Full job description
Lead and supervise the local data engineering team responsible for data platform and analytics/reporting. Manage team capacity, hiring, and define team charters aligned with the data platform roadmap. Foster a high-performance culture across distributed teams in Bangalore and EU, including onboarding and operational discipline. Plan and execute multi-quarter initiatives such as data lakehouse re-architecture, CTV late-impression pipeline, and CI/CD automation. Refine engineering processes including code quality, infrastructure-as-code, deployment automation, observability, and cost optimization. Collaborate with Product Management, Backend Engineering, Data Science, and Analytics to align platform roadmap with business goals. Maintain 50-70% hands-on technical involvement including pipeline design, feature delivery, code review, tech debt resolution, and production troubleshooting. Manage production responsibilities such as monitoring, alerts, on-call support, incident management, and postmortems. Design scalable data systems for high-throughput streaming and batch pipelines while managing infrastructure costs. Requires minimum 8 years in Data Engineering with leadership experience, strong AdTech knowledge, expert Python and SQL skills, hands-on experience with Apache Spark, Flink, Kafka, Google Cloud stack, Terraform, CI/CD tools like ArgoCD, and familiarity with lakehouse/open table formats and BI tools.
What you'll do
- Provide full supervision and support for the local data engineering pod, spanning both data-platform and analytics/reporting work streams
- Manage team capacity, lead hiring to backfill and grow the pod, and define team charters supporting a fast-evolving data platform roadmap
- Foster a high-performance culture across a distributed organization (Bangalore and EU), onboarding recent hires and establishing operational discipline (on-call/IRM, runbooks, incident response)
- Plan and execute multi-quarter initiatives with many interdependencies (e.g., data lakehouse re-architecture using Apache Flink on GKE, dedicated CTV late-impression pipeline, CI/CD automation with ArgoCD) ensuring alignment with architectural standards
- Refine engineering processes, methodologies, and technical standards: code quality, infrastructure-as-code, deployment automation, observability, and cost optimization
- Partner closely with Product Management, Backend Engineering, Data Science, and Analytics to align platform roadmap with business objectives
- Stay active in technical implementation (50-70% hands-on): designing pipelines, delivering new product features, reviewing architecture and code, addressing tech debts, troubleshooting production data issues
- Undertake production responsibilities including setting up monitoring and alerts, on-call support, troubleshooting, incident management and communication, and postmortem
- Independently design maintainable and scalable data systems for complex, high-throughput streaming and batch pipelines while managing infrastructure costs
Requirements
- Minimum of 8 years of experience in Data Engineering, with a focus on leadership in technical settings
- AdTech industry expertise (specifically programmatic auctions and high-volume bid/impression/RTB data) is strongly preferred
- Expert knowledge of Python and SQL
- Deep, hands-on experience with distributed data processing and streaming: Apache Spark / Spark Streaming and Apache Flink
- Strong experience operating high-throughput event streaming and messaging with Apache Kafka
- Proficiency with the Google Cloud data stack (BigQuery, GKE, GCS, Dataflow/Vertex AI) and infrastructure-as-code (Terraform)
- Experience building and operating CI/CD for data pipelines (e.g., ArgoCD, containerized/Docker deployments)
- Working knowledge of lakehouse and open table formats (e.g., Iceberg/Delta) and the trade-offs between real-time and batch architectures (latency, cost, governance)
- Familiarity with analytics/OLAP and BI tooling (e.g., Druid, Looker, Rill/Turnilo) and dimensional data modeling
Tech stack
PythonSQLApache SparkSpark StreamingApache FlinkApache KafkaGoogle CloudBigQueryGKEGCSDataflowVertex AITerraformArgoCDDockerIcebergDeltaDruidLookerRillTurnilo
Benefits
Apollo Health Check-upsMedical Insurance for employee and familyINR 4100/month Personalized Benefits Platform (Travel, entertainment, food, fitness, healthcare)Office lunches 3 times a week through ZomatoOffice sweets, snacks, and beveragesAnnual training budget and internal webinarsGerman/English language classes19 paid vacation days + 3 Wellness days plus public holidaysMonthly Work&Travel raffle to work from global offices (after 2 years of employment)