data engineeringleadershippythonsqlapache sparkapache flinkapache kafkagoogle cloudbigqueryterraformargoCDdockerlakehouseicebergdeltaadtechprogrammaticstreamingci/cddata pipelines
Key details
Salary
Not specified
Employment type
Full-time
Seniority
Lead
Years experience
5-10
Location
Bengaluru, India
Full job description
Lead and supervise the local data engineering team supporting data-platform and analytics/reporting. Manage team capacity, hiring, and define team charters aligned with the evolving data platform roadmap. Foster a high-performance culture across distributed teams in Bangalore and EU, including onboarding and operational discipline. Plan and execute multi-quarter initiatives such as data lakehouse re-architecture using Apache Flink on GKE, CTV late-impression pipeline, and CI/CD automation with ArgoCD. Refine engineering processes including code quality, infrastructure-as-code, deployment automation, observability, and cost optimization. Collaborate with Product Management, Backend Engineering, Data Science, and Analytics to align platform roadmap with business goals. Maintain 50-70% hands-on technical involvement designing pipelines, delivering features, reviewing architecture and code, addressing technical debt, and troubleshooting production issues. Manage production responsibilities including monitoring, on-call support, incident management, and postmortems. Design scalable data systems for high-throughput streaming and batch pipelines while managing infrastructure costs. Requires minimum 8 years in Data Engineering with leadership experience, strong AdTech knowledge, expert Python and SQL skills, experience with Apache Spark, Flink, Kafka, Google Cloud stack, Terraform, ArgoCD, Docker, lakehouse/open table formats, and BI tools.
What you'll do
Similar jobs
More roles worth a look
Related opportunities based on specialty and working model so candidates can keep momentum.
Provide full supervision and support for the local data engineering pod, spanning both data-platform and analytics/reporting work streams
Manage team capacity, lead hiring to backfill and grow the pod, and define team charters supporting a fast-evolving data platform roadmap
Foster a high-performance culture across a distributed organization (Bangalore and EU), onboarding recent hires and establishing strong operational discipline (on-call/IRM, runbooks, incident response)
Plan and execute multi-quarter initiatives with many interdependencies (e.g., data lakehouse re-architecture with Apache Flink on GKE, dedicated CTV late-impression pipeline, CI/CD automation with ArgoCD) ensuring alignment with architectural standards
Refine engineering processes, methodologies, and technical standards: code quality, infrastructure-as-code, deployment automation, observability, and cost optimization
Partner closely with Product Management, Backend Engineering, Data Science, and Analytics to align platform roadmap with business objectives
Stay active in technical implementation (50-70% hands-on): designing pipelines, delivering new product features, reviewing architecture and code, addressing tech debts, troubleshooting production data issues
Undertake production responsibilities including monitoring and alerts setup, oncall support, troubleshooting, incident management and communication, and postmortem
Independently design maintainable and scalable data systems for complex, high-throughput streaming and batch pipelines while managing infrastructure cost
Requirements
Minimum of 8 years of experience in Data Engineering, with a focus on leadership in technical settings
AdTech industry expertise (specifically programmatic auctions and high-volume bid/impression/RTB data) is strongly preferred
Expert knowledge of Python and SQL
Deep, hands-on experience with distributed data processing and streaming: Apache Spark / Spark Streaming and Apache Flink
Strong experience operating high-throughput event streaming and messaging with Apache Kafka
Proficiency with the Google Cloud data stack (BigQuery, GKE, GCS, Dataflow/Vertex AI) and infrastructure-as-code (Terraform)
Experience building and operating CI/CD for data pipelines (e.g., ArgoCD, containerized/Docker deployments)
Working knowledge of lakehouse and open table formats (e.g., Iceberg/Delta) and the trade-offs between real-time and batch architectures (latency, cost, governance)
Familiarity with analytics/OLAP and BI tooling (e.g., Druid, Looker, Rill/Turnilo) and dimensional data modeling
Apollo Health Check-upsMedical Insurance for employee and familyINR 4100/month Personalized Benefits Platform (Travel, entertainment, food, fitness, healthcare)Office lunches 3 times a week through ZomatoOffice sweets, snacks, and beveragesAnnual training budget and internal webinarsGerman/English language classes19 paid vacation days + 3 Wellness days plus public holidaysMonthly Work&Travel raffle to work from global offices (after 2 years of employment)
Apply now
Ready to take the next step in your career? Click the button below to continue to the application process.
