data engineeringleadershippythonsqlapache sparkapache flinkapache kafkagoogle cloudbigquerygketerraformargocddockerlakehousedeltaicebergadtechprogrammaticstreamingci/cddata pipelinesetlanalyticsbidimensional modeling
Key details
Salary
Not specified
Employment type
Full-time
Seniority
Lead
Years experience
5-10
Location
Bengaluru, India
Full job description
Lead the local data engineering team supporting data-platform and analytics/reporting. Manage team capacity, hiring, and define team charters aligned with the data platform roadmap. Foster a high-performance culture across distributed teams in Bangalore and EU, including onboarding and operational discipline. Plan and execute multi-quarter initiatives such as data lakehouse re-architecture using Apache Flink on GKE, CTV late-impression pipeline, and CI/CD automation with ArgoCD. Refine engineering processes including code quality, infrastructure-as-code, deployment automation, observability, and cost optimization. Collaborate with Product Management, Backend Engineering, Data Science, and Analytics to align platform roadmap with business goals. Maintain 50-70% hands-on technical involvement including pipeline design, feature delivery, architecture/code reviews, tech debt resolution, and production troubleshooting. Handle production responsibilities including monitoring, alerts, on-call support, incident management, and postmortems. Design scalable, maintainable data systems for high-throughput streaming and batch pipelines while managing infrastructure costs. Requires minimum 8 years experience in data engineering leadership, strong Python and SQL skills, expertise in Apache Spark, Flink, Kafka, Google Cloud data stack, Terraform, ArgoCD, Docker, lakehouse/open table formats, and familiarity with analytics/BI tools. AdTech experience preferred.
What you'll do
Similar jobs
More roles worth a look
Related opportunities based on specialty and working model so candidates can keep momentum.
Provide full supervision and support for the local data engineering pod, spanning data-platform and analytics/reporting work streams
Manage team capacity, lead hiring to backfill and grow the pod, define team charters supporting data platform roadmap
Foster high-performance culture across distributed organization (Bangalore and EU), onboard new hires, establish operational discipline (on-call/IRM, runbooks, incident response)
Plan and execute multi-quarter initiatives with interdependencies (data lakehouse re-architecture with Apache Flink on GKE, dedicated CTV late-impression pipeline, CI/CD automation with ArgoCD) ensuring alignment with architectural standards
Partner with Product Management, Backend Engineering, Data Science, and Analytics to align platform roadmap with business objectives
Stay active in technical implementation (50-70% hands-on): design pipelines, deliver new features, review architecture and code, address tech debt, troubleshoot production data issues
Undertake production responsibilities including monitoring, alerts, on-call support, troubleshooting, incident management and communication, postmortem
Independently design maintainable and scalable data systems for complex, high-throughput streaming and batch pipelines, manage infrastructure cost
Requirements
Minimum of 8 years of experience in Data Engineering, with a focus on leadership in technical settings
AdTech industry expertise (specifically programmatic auctions and high-volume bid/impression/RTB data) is strongly preferred
Expert knowledge of Python and SQL
Deep, hands-on experience with distributed data processing and streaming: Apache Spark / Spark Streaming and Apache Flink
Strong experience operating high-throughput event streaming and messaging with Apache Kafka
Proficiency with the Google Cloud data stack (BigQuery, GKE, GCS, Dataflow/Vertex AI) and infrastructure-as-code (Terraform)
Experience building and operating CI/CD for data pipelines (e.g., ArgoCD, containerized/Docker deployments)
Working knowledge of lakehouse and open table formats (e.g., Iceberg/Delta) and the trade-offs between real-time and batch architectures (latency, cost, governance)
Familiarity with analytics/OLAP and BI tooling (e.g., Druid, Looker, Rill/Turnilo) and dimensional data modeling
Apollo Health Check-upsMedical Insurance for employee and familyINR 4100/month Personalized Benefits Platform (Travel, entertainment, food, fitness, healthcare)Office lunches 3 times a week through ZomatoOffice sweets, snacks, and beveragesAnnual training budget and internal webinarsGerman/English language classes19 paid vacation days + 3 Wellness days plus public holidaysMonthly Work&Travel raffle to work from global offices (after 2 years of employment)
Apply now
Ready to take the next step in your career? Click the button below to continue to the application process.
