Senior Big Data Engineer

Key details

Full job description

Seeking a Senior Software Engineer for Epsilon’s Data Practice focused on TotalSource Plus demographic and lifestyle products. Responsibilities include designing, developing, and maintaining scalable data processing solutions using Python and Apache Spark on Databricks, optimizing Spark jobs, building fault-tolerant data pipelines with monitoring, leveraging AWS cloud services, developing SQL queries, applying data modeling best practices, using Git for source control, collaborating with cross-functional teams, extracting insights from large datasets, and mentoring junior engineers. Requirements include 5+ years of software development experience in scalable distributed environments, proficiency in Scala, Python, or Java, experience with Apache Spark, Hadoop, Hive, cloud platforms (AWS preferred), Kubernetes, Docker, Airflow, strong problem-solving and communication skills, BA/BS in Computer Science or related field, machine learning experience, and preferred certifications in AWS, Databricks, or Spark. Benefits include flexible time off, paid holidays and sick time, parental leave, childcare and elder care assistance, adoption assistance, health coverage, 401(k), tuition assistance, commuter benefits, professional development, employee recognition, charitable donation matching, and health coaching.