AdTechTalent
Data Science14 days agoOn-site

The Trade Desk

Senior Software Engineer - Back-end/Data Pipeline - VenturaOS

data engineeringbig dataapache sparkkafkajavascalapythonmachine learningmlopspersonalizationcontent recommendationsad-techstreamingtvosdata pipelinesdata warehousingsnowflakeverticadatabrickscloudawsgcpazurekubernetesdistributed systemsdata governancenosqlrelational databasesrecommendation systemssearch technologiese-commerce

Key details

Salary

$125K – $229K

Employment type

Full-time

Seniority

Senior

Years experience

5-10

Location

San Jose, United States

Full job description

The Trade Desk is seeking a senior full-time data engineer based in San Jose, California. The role focuses on building and maintaining scalable data pipelines and backend infrastructure for the Ventura OS project, emphasizing streaming, TVOS, and e-commerce data ecosystems. Responsibilities include designing data workflows, creating APIs, collaborating with data scientists, and optimizing data storage and retrieval. Candidates should have 7+ years of software development experience with at least 5 years in data engineering, expertise in data warehousing (Vertica, Snowflake, Databricks), big data technologies (Apache Spark, Kafka), programming languages (Java, Scala, Python), cloud platforms (AWS, GCP, Azure), and containerization (Kubernetes). Strong knowledge of data governance, machine learning pipelines, recommendation systems, and distributed systems is required. Benefits include comprehensive healthcare, retirement plans, disability coverage, life insurance, tuition reimbursement, parental leave, paid sick time and vacation, paid holidays, stock purchase plans, and variable incentives. Salary range is $124,900 to $228,900 USD.

What you'll do

  • Design, build, and maintain data pipelines for processing content catalog data
  • Develop efficient data processing workflows handling large-scale data with optimal performance
  • Create robust APIs to securely expose data services
  • Collaborate with data scientists to integrate models and algorithms in production
  • Build scalable data integration solutions connecting various data sources and destinations
  • Implement monitoring and observability for data pipelines to ensure reliability and performance
  • Optimize data storage and retrieval mechanisms
  • Contribute to architectural decisions around data infrastructure
  • Work closely with cross-functional teams to translate business requirements into technical solutions
  • Apply best practices for data engineering including data quality checks, error handling, and recovery mechanisms

Requirements

  • Extensive experience with data warehousing solutions like Vertica, Snowflake, or Databricks
  • Strong knowledge of data governance and data quality frameworks
  • Hands-on experience with machine learning pipelines and MLOps in personalization, content recommendations, ad-tech solutions, or e-commerce optimization within streaming or TVOS platforms
  • Expertise in content metadata processing and management within streaming, TVOS, or e-commerce environments
  • Familiarity with entertainment catalog systems, taxonomy structures, and content discovery workflows
  • Experience with recommendation systems or search technologies using AI-driven content surfacing
  • Strong communication skills across technical and non-technical stakeholders
  • Solid foundation in computer science and engineering fundamentals
  • Deep experience with distributed systems and data processing frameworks handling large-scale datasets
  • Proficiency in one or more programming languages such as Java, Scala, or Python
  • Strong expertise in big data technologies such as Apache Spark and Kafka
  • Understanding of data modeling concepts and experience with relational and NoSQL databases
  • Hands-on experience with cloud platforms like AWS, GCP, or Azure and containerization technologies such as Kubernetes
  • Proven ability to lead and complete projects with high technical ambiguity while mentoring peers
  • 7+ years of software development experience with at least 5 years specializing in data engineering or pipeline development
  • Bachelor’s or Master’s degree in Computer Science, Engineering, or related field or equivalent experience

Tech stack

Apache SparkVerticaSnowflakeDatabricksJavaScalaPythonKafkaAWSGCPAzureKubernetes

Benefits

Comprehensive healthcare (medical, dental, vision) with premiums paid in full for employees and dependentsRetirement benefits including 401k plan and company matchShort and long-term disability coverageBasic life insuranceWell-being benefitsTuition reimbursement for certain expensesParental leaveSick time of 1 hour per 30 hours workedVacation time up to 120 hours in first year and 160 hours thereafterAround 13 paid holidays per yearEmployee Stock Purchase Plan with discounted stock purchaseVariable compensation-based incentives and commissions depending on role

Apply now

Ready to take the next step in your career? Click the button below to continue to the application process.

Continue to applicationBrowse more jobs

Similar jobs

More roles worth a look

Related opportunities based on specialty and working model so candidates can keep momentum.