Senior Data Engineer - Big Data

data engineeringbig dataapache sparketleltgcpawsbigquerys3gcsemrairflowkafkadata warehousedata lakemedallion architecturelakehouseapache icebergdelta lakescalapythonjavaflinkhivetrinostreamingbatch processing

Key details

Salary

Not specified

Employment type

Full-time

Seniority

Senior

Years experience

5-10

Location

Netanya, Israel

Full job description

Teads is seeking a Senior Big Data Engineer with 5+ years of experience to design, build, and maintain scalable data pipelines and infrastructure. The role involves working with Apache Spark and distributed data processing frameworks, managing cloud data infrastructure on GCP or AWS, and improving data warehouse and data lake solutions. Candidates must have production experience with large-scale batch and streaming pipelines, Kafka, and strong knowledge of data platform best practices including Medallion Architecture. Nice to have experience with lakehouse table formats such as Apache Iceberg or Delta Lake. The position is full-time with a hybrid work model based in Netanya, Israel.

What you'll do

Design, build, and maintain ETL/ELT data pipelines using Apache Spark to ingest, process, and transform large-scale datasets
Architect and manage data infrastructure primarily on Google Cloud Platform (GCP) or Amazon Web Services (AWS), including BigQuery, S3, GCS, EMR, and AirFlow
Improve and manage data warehouse and data lake solutions ensuring data quality, consistency, and accessibility for business intelligence and machine learning
Collaborate with cross-functional teams to understand data needs and implement solutions supporting new product features and business initiatives
Implement monitoring, alerting, and logging systems to maintain data pipeline health and ensure data accuracy

Requirements

5+ years of data engineering experience building and operating production data pipelines at scale (TB+ datasets, hourly/daily batch or streaming workloads)
Hands-on production experience with Apache Spark and distributed data processing frameworks such as Flink, Hive, or Trino
Strong understanding of large-scale batch and streaming pipelines, including performance tuning and troubleshooting
Ability to debug and ship production Spark code in Scala, Python, or Java
Production experience building and operating data solutions on GCP or AWS, including cloud-native services such as BigQuery, Dataproc, GCS, S3, EMR, or Redshift
Production experience with Kafka or Kafka-compatible streaming platforms, including development, operation, and troubleshooting of real-time data pipelines
Strong understanding of data warehouse and data lake concepts, including Medallion Architecture (Bronze, Silver, Gold) and data platform best practices
Nice to have: Production experience with a lakehouse table format (Apache Iceberg or Delta Lake), including schema evolution and table maintenance

Tech stack

ScalaTypeScriptApache SparkFlinkHiveTrinoPythonJavaGoogle Cloud PlatformAmazon Web ServicesBigQueryS3GCSEMRAirFlowKafkaApache IcebergDelta Lake

Benefits

Hybrid working model (3 days per week in the office)Nearby parking availableMentorship program and internal learning toolsPet friendly officeHappy hoursFully stocked kitchen