AdTechTalent
Data Science6 days agoOn-site

Smadex

Data Engineering Team Lead

data engineeringleadershipsparkscalapythonapache airflowawsbig datafeature storedistributed systemssqladtechreal-time biddingrtbdspdockerkubernetesdata qualitymachine learning

Key details

Salary

Not specified

Employment type

Full-time

Seniority

Lead

Years experience

5-10

Location

Austin, United States; New York, US

Full job description

Smadex is seeking a Data Engineering Team Lead with expertise in big data architectures and leadership experience. The role is full-time, based in Austin, TX or New York, NY. Responsibilities include leading two sub-teams (Data Quality and Feature Engineering), designing scalable data pipelines for real-time ML inference, establishing data quality frameworks, enforcing data governance, overseeing feature store development, coordinating pipeline strategies with infrastructure, ensuring operational excellence, and collaborating with executive leadership. Requirements include 6+ years in software or data engineering with 2+ years in leadership, advanced Spark and SQL skills, AWS experience, strong debugging and communication skills. Preferred experience includes AdTech, RTB/DSP ecosystems, feature stores, containerization (Docker, Kubernetes), and a master's degree in a relevant field. The tech stack includes Python, Scala, Bash, Spark, MySQL, Redis, AWS, Jenkins, Docker, Airflow, Git, Jira, and Notion. Benefits include competitive compensation, medical/dental/vision plans, career growth opportunities, and potential travel to Barcelona HQ. Visa sponsorship is not provided.

What you'll do

  • Lead and scale two specialized sub-teams (Data Quality and Feature Engineering), managing performance, career development, and hiring
  • Design scalable, highly available data pipelines bridging big data storage and real-time machine learning inference
  • Establish rigorous standards for data observability, implementing automated monitoring and alerting for complex data ingestion services
  • Define and enforce policies for data privacy, security, and lifecycle management, ensuring compliance with relevant regulations and company standards
  • Oversee the development and optimization of high-throughput features using Scala to aggregate data for Redis Cluster
  • Coordinate with the infrastructure team ensuring the execution of complex data workflows and DAGs using Apache Airflow
  • Monitor global deployments and ensure the stability, reliability, and performance of production systems following a 'you build it, you run it' philosophy
  • Partner with the Chief Data Officer to align technical strategies with business objectives and Machine Learning roadmaps
  • Collaborate closely with the CTO for technical solutions and strategy, and partner with the CPO on product integration and transversal projects

Requirements

  • 6+ years of experience in software or data engineering, with at least 2 years in a formal leadership or management role building high-throughput distributed systems
  • Advanced knowledge of Spark and ability to conduct deep code reviews
  • Proven track record of designing and optimizing complex batch and streaming pipelines using Apache Spark
  • Advanced SQL knowledge and experience with cloud environments (AWS), particularly EMR, EC2, Athena, and S3
  • Strong ability to debug complex distributed systems and drive root-cause analysis for production issues
  • Excellent communication skills in English, with the ability to translate technical complexity for stakeholders

Tech stack

PythonScalaBashSparkMySQLRedisAWSJenkinsDockerAirflowGitJiraNotion

Benefits

Be part of a leading, fast-growing, innovative company shaping the future of mobile advertisingJoin a highly motivated and young teamPossibility of traveling to the Barcelona HQ for collaboration and team-building activities after your first yearGreat compensation package tailored to the U.S. marketMedical, dental, and vision benefits plansExposure to leading global app publishers and media partners in the digital advertising industryLearning and training opportunities to grow your career

Apply now

This MVP uses a placeholder application flow. In production, this section can connect to an external apply URL or a native application form.

Continue to applicationBrowse more jobs

Similar jobs

More roles worth a look

Related opportunities based on specialty and working model so candidates can keep momentum.