M30 - Data Engineer

Role Overview We are seeking an experienced Senior Data Engineer to design, develop, deploy and maintain scalable, reliable and secure data pipelines and reusable data products using AWS and Databricks. The successful candidate will provide technical leadership across data ingestion, transformation, modelling, quality, orchestration, monitoring and performance optimisation to support analytics, reporting and AI/ML workloads. This is a hands-on technical role requiring strong expertise in cloud-based data engineering, lakehouse architecture, production pipeline operations and engineering best practices. Key Responsibilities Lead the design, development and implementation of scalable batch and streaming data pipelines using AWS and Databricks. Design and maintain lakehouse architectures using Bronze, Silver and Gold data layers, Delta Lake and reusable data models. Develop reliable data ingestion processes from databases, APIs, files and event streams, including incremental loading and Change Data Capture (CDC). Implement automated data quality checks, schema validation, reconciliation and anomaly detection to ensure data accuracy and consistency. Build and manage workflow orchestration using Databricks Workflows, Lakeflow and relevant AWS services. Implement CI/CD pipelines, automated testing and Infrastructure as Code (IaC) to support efficient development and deployment. Establish data governance practices, including metadata management, data lineage, access controls and dat