AdTechTalent
Data Science314 days agoRemote

Kargo

Senior Data Engineer

pythonsparkairflowapache icebergawseksdockerkubernetesargocdprometheussnowflakeclickhousekafkaflinkaerospikeetldata pipelinesci/cdprivacygdprccpadata engineeringad tech

Key details

Salary

Not specified

Employment type

Full-time

Seniority

Senior

Years experience

5-10

Location

London, England, United Kingdom

Full job description

Senior Data Engineer role to build and scale data infrastructure for targeting, identity, and measurement. Responsibilities include optimizing ETL/ELT pipelines, onboarding new datasets, adopting CI/CD practices, implementing privacy-compliant data methodologies (GDPR/CCPA), and maintaining thorough technical documentation. Required skills: Python, Spark, Airflow, Apache Iceberg, AWS (EKS, Docker, Kubernetes), ArgoCD, Prometheus, Snowflake or ClickHouse, and knowledge of ad tech privacy and targeting. Preferred skills: Kafka, Flink, Aerospike. Remote role based in London, UK.

What you'll do

  • Build and scale data infrastructure powering targeting, identity, and measurement capabilities
  • Implement and optimize ETL/ELT pipelines ensuring operational reliability with documented SLAs
  • Onboard new datasets to power targeting capabilities and meet defined KPIs
  • Adopt and maintain CI/CD practices to accelerate team velocity
  • Implement privacy-compliant data methodologies meeting GDPR/CCPA standards
  • Create thorough technical documentation including implementation docs and runbooks
  • Take operational responsibility for systems including monitoring and iterative improvements
  • Collaborate actively in design and brainstorming sessions
  • Communicate technical decisions clearly to technical and non-technical stakeholders
  • Write and maintain documentation that supports onboarding and knowledge sharing

Requirements

  • Deep proficiency in Python and Spark for building and optimizing large-scale data pipelines
  • Hands-on experience with Airflow for orchestrating complex data workflows
  • Experience with Apache Iceberg for table format management at scale
  • Extensive AWS experience including EKS, Docker, and Kubernetes
  • CI/CD fluency with ArgoCD, automated testing pipelines, and monitoring via Prometheus
  • Familiarity with analytical warehousing in Snowflake or ClickHouse including SQL optimization and cost management
  • Working knowledge of identity, privacy, and targeting methodologies in ad tech
  • Preferred experience with real-time streaming technologies such as Kafka and Flink
  • Preferred familiarity with Aerospike or similar low-latency key-value stores

Tech stack

PythonSparkAirflowApache IcebergAWSEKSDockerKubernetesArgoCDPrometheusSnowflakeClickHouseKafkaFlinkAerospike

Apply now

This MVP uses a placeholder application flow. In production, this section can connect to an external apply URL or a native application form.

Similar jobs

More roles worth a look

Related opportunities based on specialty and working model so candidates can keep momentum.