Middle Data Engineer

About the team We are a focused engineering team of 5 to 9 people at a multi-product tech SaaS company. We work across personalization, recommendation, fraud detection, and some GenAI, through real-time, near-real-time, and batch pipelines. The team is a blend of Python backend engineers, ML engineers, QA (manual and automation), a data engineer, and data scientists. Our shared mandate is to take experimentation from our data scientists and turn it into robust, production-grade systems, either as standalone services or integrated into wider platforms. The role You will build the data pipelines that power our experimentation and production systems, the backbone behind fraud detection, recommendations, and personalization. You will work primarily with Kafka (sometimes Flink), Spark on Databricks, and Airflow, modelling and optimizing data for both experimentation and production serving, and partnering closely with data scientists and backend engineers on data contracts and reliability. Responsibilities • Build and operate batch and streaming data pipelines (structured & unstructured data) • Work primarily with Kafka, Spark on Databricks, and Airflow • Model and optimize data for both experimentation and production serving • Partner with data scientists and backend engineers on data contracts and reliability • Own pipeline quality: testing, monitoring, and performance • Take part in the teams on-call rotation for the pipelines you build and operate Requirements • 3+ years of