We are seeking a skilled Data Engineer with strong expertise in modern data technologies to design and develop scalable, high-performance data platforms and pipelines. You’ll work closely with product, analytics, and engineering teams to build robust data architectures, optimize data workflows, and deliver reliable, high-quality data solutions across multiple projects.

• Work closely with source system teams and reporting teams to gather, analyze, and translate data requirements into scalable data pipelines.

• Design, develop, and maintain robust ETL/ELT pipelines using PySpark, SQL, and Delta Lake on Azure Databricks.

• Ingest and integrate data from multiple sources including MariaDB, Azure Event Hubs, APIs, and flat file systems.

• Build and optimize data models across bronze, silver, and gold layers using Delta tables.

• Ensure data reliability, accuracy, and freshness through monitoring, testing, and validation.

• Apply and enforce data governance, security, and compliance policies throughout the pipeline lifecycle.

• Collaborate with analysts, data scientists, and engineers to enable downstream data usage.

• Promote engineering excellence through code reviews, documentation, CI/CD, and automation.

• Mentor junior engineers and provide guidance on design and best practices.

• Explore emerging technologies such as Structured Streaming, and support their adoption where beneficial.

• Strong hands-on skills in Azure Databricks, PySpark, and SQL.

• Solid experience working with Delta Lake and managing Delta tables for large-scale data processing.

• Proficiency in data modeling, distributed data processing, and performance tuning.

• Familiarity with Structured Streaming is a plus, but not mandatory.

• Deep understanding of cloud data architectures (e.g., Data Lake, Lakehouse).

• Understanding of data security principles, including encryption, access controls, masking, and audit logging.

• Awareness of data governance and compliance frameworks (e.g., GDPR, data lineage, access policies).

• Familiarity with dbt (Data Build Tool) is a plus, but not required.

• Strong awareness of data quality frameworks, observability tools, and validation strategies.

• Excellent communication and collaboration skills to engage with cross-functional teams.

• Ability to lead by example and promote best practices across the engineering team.

• 5+ years of experience in data engineering or similar roles.

• At least 1 year in a senior or technical lead capacity.

• Proven experience with Azure Databricks, Delta Lake, and Spark-based data pipelines.

• Experience integrating data from multiple sources such as MariaDB, Event Hubs, APIs, or file systems.

• Exposure to version control (e.g., Git), CI/CD pipelines, and Agile methodologies.

• Familiarity with Structured Streaming and real-time data processing is a plus.

• Experience in fintech, broking, or financial services domains is an advantage.