AdTechTalent
Data Science12 months agoRemote

Kargo

Senior Data Engineer

pythonsparkairflowapache icebergawseksdockerkubernetesargocdprometheussnowflakeclickhousekafkaflinkaerospikesqldata engineeringetlci/cdprivacygdprccpaad techdata pipelines

Key details

Salary

Not specified

Employment type

Full-time

Seniority

Senior

Years experience

5-10

Location

London, United Kingdom

Full job description

Senior Data Engineer role to build and scale data infrastructure for targeting, identity, and measurement. Responsibilities include optimizing ETL/ELT pipelines, onboarding new datasets, adopting CI/CD practices, implementing privacy-compliant data methodologies, and maintaining thorough technical documentation. Required skills: Python, Spark, Airflow, Apache Iceberg, AWS (EKS, Docker, Kubernetes), ArgoCD, Prometheus, Snowflake or ClickHouse, SQL, and knowledge of ad tech privacy standards (GDPR/CCPA). Preferred skills: Kafka, Flink, Aerospike. Remote role based in London, UK.

What you'll do

  • Build and scale data infrastructure powering targeting, identity, and measurement capabilities
  • Implement and optimize ETL/ELT pipelines ensuring operational reliability and documented SLAs
  • Onboard new datasets to power targeting capabilities with defined KPIs
  • Adopt CI/CD practices to accelerate team velocity using ArgoCD-based deployment pipelines
  • Implement and document privacy-compliant data methodologies meeting GDPR/CCPA standards
  • Create thorough technical documentation including implementation docs, runbooks, and best practices
  • Take operational responsibility for systems including monitoring, iteration, and improvement
  • Collaborate actively in design and brainstorming sessions
  • Communicate technical decisions clearly to both technical and non-technical stakeholders
  • Write and maintain documentation that supports onboarding and knowledge sharing

Requirements

  • Deep proficiency in Python and Spark for building and optimizing large-scale data pipelines
  • Strong grasp of data structuring for performance and efficiency
  • Hands-on experience with Airflow for orchestrating complex data workflows
  • Experience with Apache Iceberg for table format management at scale
  • Extensive AWS experience including EKS, Docker, and Kubernetes
  • Comfortable owning infrastructure deployment and scaling decisions
  • CI/CD fluency including ArgoCD, automated testing pipelines, and monitoring via Prometheus
  • Familiarity with analytical warehousing in Snowflake or ClickHouse including SQL optimization, cost management, and performance tuning
  • Working knowledge of identity, privacy, and targeting methodologies in ad tech
  • Preferred: Experience with real-time streaming technologies such as Kafka and Flink
  • Preferred: Familiarity with Aerospike or similar low-latency key-value stores

Tech stack

PythonSparkAirflowApache IcebergAWSEKSDockerKubernetesArgoCDPrometheusSnowflakeClickHouseKafkaFlinkAerospikeSQL

Apply now

Ready to take the next step in your career? Click the button below to continue to the application process.

Similar jobs

More roles worth a look

Related opportunities based on specialty and working model so candidates can keep momentum.