AdTechTalent
Engineering1 month agoHybrid

Epsilon

Senior Software Engineer

pythonapache sparkdatabricksawssqlscalajavahadoophivekubernetesdockerairflowmachine learningbig datadata pipelinesspark optimizationclouddata engineering

Key details

Salary

$89K – $165K

Employment type

Full-time

Seniority

Senior

Years experience

5-10

Location

Westminster, United States

Full job description

Seeking a Senior Software Engineer to join Epsilon’s Data Practice team focused on TotalSource Plus demographic and lifestyle products. Responsibilities include designing, developing, and maintaining scalable data processing solutions using Python and Apache Spark on Databricks, optimizing Spark jobs, building fault-tolerant data pipelines with monitoring and alerting, leveraging AWS cloud services, developing and optimizing SQL queries, applying design patterns for data modeling and distributed systems, using Git for source control, collaborating with cross-functional teams, extracting insights from large datasets, and mentoring junior engineers. Requirements include 5+ years in scalable distributed software development, proficiency in Scala, Python, or Java, experience with Apache Spark, Hadoop, Hive, AWS, Kubernetes, Docker, MWAA/Airflow, strong problem-solving and communication skills, a BA/BS in Computer Science or related field, machine learning experience, and preferred certifications in AWS, Databricks, or Spark.

What you'll do

  • Design, develop, and maintain scalable data processing solutions across on-premises and cloud environments using Python and Apache Spark on Databricks
  • Optimize and fine-tune Spark jobs for performance including resource utilization, shuffling, partitioning, and caching
  • Design and implement scalable, fault-tolerant data pipelines with monitoring, alerting, and logging
  • Leverage AWS cloud services to build and manage data pipelines and distributed processing workloads
  • Develop and optimize SQL queries across relational and data warehouse systems
  • Apply design patterns and best practices for efficient data modeling, partitioning, and distributed system performance
  • Use Git for source control and maintain strong unit and integration testing practices
  • Collaborate with Product Owners, partners, and cross-functional teams to translate business requirements into technical solutions
  • Extract actionable insights from large datasets to support data-driven decision-making
  • Mentor junior engineers, conduct code reviews, and contribute to engineering best practices and standards

Requirements

  • 5+ years of software development experience in scalable, distributed, or multi-node environments
  • Proficient in Scala, Python, or Java
  • Significant experience with Apache Spark and exposure to Hadoop, Hive, and related big data technologies
  • Experience with cloud platforms, preferably AWS
  • Exposure to Kubernetes, Docker, and MWAA/Airflow
  • Strong problem-solving skills and ability to deliver results end-to-end
  • Consultative mindset with strong communication and relationship-building skills
  • Collaborative team player with eagerness to learn and contribute
  • BA/BS in Computer Science or related discipline
  • Experience in Machine Learning including model development, feature engineering, or integrating ML workflows
  • Experience with Databricks components such as Notebooks, Delta Lake, Jobs, Pipelines, and Unity Catalog preferred
  • AWS, Databricks, or Spark certification is a plus

Tech stack

PythonApache SparkDatabricksAWSSQLScalaJavaHadoopHiveKubernetesDockerMWAAAirflowDelta LakeUnity CatalogGit

Benefits

Flexible time off (FTO)15 paid holidaysPaid sick timeParental/new child leaveChildcare & elder care assistanceAdoption assistanceComprehensive health coverage401(k)Tuition assistanceCommuter benefitsProfessional developmentEmployee recognitionCharitable donation matchingHealth coaching and counseling

Apply now

Ready to take the next step in your career? Click the button below to continue to the application process.

Continue to applicationBrowse more jobs

Similar jobs

More roles worth a look

Related opportunities based on specialty and working model so candidates can keep momentum.