AdTechTalent
Data Science8 months agoOn-site

Appodeal, Inc.

Senior Data Engineer

pythonpandaspysparkdatabricksapache sparkdelta lakedelta live tablesclickhouseapache druiddagsterairflowkafkaspark structured streamingawsdata engineeringetldata pipelinesml data platformolapstreaming

Key details

Salary

Not specified

Employment type

Full-time

Seniority

Mid-level

Years experience

3-5

Location

Barcelona, Spain

Full job description

Appodeal is seeking a Data Engineer for the Mobile Growth Platform UA team in Barcelona. Responsibilities include designing and building scalable data pipelines using Databricks (Apache Spark, Delta Lake, Delta Live Tables), managing data architecture integrating Databricks with OLAP engines (ClickHouse/Druid), developing and orchestrating data workflows, optimizing performance of data workflows and databases, developing internal automation tools, and collaborating with product, MLOps, and data science teams. Requirements: 3-5+ years in data engineering, strong Python and data frameworks (Pandas, PySpark), experience with data lakes and warehouses, data orchestration tools (Dagster, Airflow), streaming technologies (Kafka, Spark structured streaming), cloud platforms (AWS S3, EC2), OLAP databases (ClickHouse, Apache Druid), strong problem-solving skills, and based in or willing to relocate to Barcelona. Nice to have: experience with Databricks platform and building ML data platforms.

What you'll do

  • Design and build scalable data pipelines using Databricks platform (Apache Spark, Delta Lake, Delta Live Tables) for batch and streaming data ingestion
  • Define and maintain a scalable and secure data platform architecture integrating Databricks with OLAP engines (ClickHouse/Druid)
  • Design, develop, test, and orchestrate data workflows to streamline pipelines
  • Monitor, troubleshoot, and fine-tune data workflows and database performance (Spark job optimization, ClickHouse schema tuning, Druid data partitioning)
  • Contribute to development of internal tools for automating data processes and business workflows
  • Collaborate closely with product, MLOps and data science teams to ensure optimal data experience for ML practitioners and data consumers

Requirements

  • 3-5+ years of professional work experience in data engineering
  • Strong expertise in Python and common data frameworks (Pandas, PySpark)
  • Experience with modern data storage solutions: Data lakes (Delta tables), and data warehouses
  • Experience with data orchestration tools (Dagster, Airflow)
  • Experience with streaming technologies (Kafka, Spark structured streaming)
  • Experience with cloud platforms (AWS S3, EC2 etc.)
  • Hands-on experience with ClickHouse and/or Apache Druid for real-time analytics and low-latency data serving
  • Strong problem-solving skills and ability to excel in a fast-paced startup environment
  • Based in Barcelona or ready to relocate to Barcelona

Tech stack

PythonPandasPySparkDatabricksApache SparkDelta LakeDelta Live TablesClickHouseApache DruidDagsterAirflowKafkaSpark structured streamingAWS S3AWS EC2

Apply now

Ready to take the next step in your career? Click the button below to continue to the application process.

Continue to applicationBrowse more jobs

Similar jobs

More roles worth a look

Related opportunities based on specialty and working model so candidates can keep momentum.