AdTechTalent
Engineering1 month agoHybrid

Epsilon

Staff Software Engineer

scalaapache sparkhadooppythonsqlbig datacloudawsgcpdatabrickselk stackdockerkubernetesdata engineeringagilescrummentoring

Key details

Salary

Not specified

Employment type

Full-time

Seniority

Lead

Years experience

10+

Location

Bengaluru, India

Full job description

Lead/Principal Software Engineer role in the Epsilon Attribution/Forecasting Product Development team. Responsibilities include designing, implementing, and optimizing data processing solutions using Scala, Spark, and Hadoop in on-premises and cloud environments. Collaborate with teams to deploy big data solutions, build and maintain workflows, perform data integration and transformation, troubleshoot issues, document processes, and enhance attribution and forecasting engines. Requires 12+ years of experience in Scala and Apache Spark, strong Python skills, deep knowledge of Hadoop ecosystem, SQL proficiency, experience with DAG workflows, data warehousing, GIT, unit and integration testing, and Agile methodologies. Must have strong communication skills and ability to mentor junior staff. Experience with AWS, GCP, Databricks, ELK stack, Docker, Kubernetes, and data visualization tools is advantageous. Location: Bengaluru, Karnataka, India.

What you'll do

  • Design, implement, and optimize data processing solutions using Scala, Spark, and Hadoop
  • Collaborate with cross-functional teams to deploy big data solutions on on-premises and cloud infrastructure
  • Build, schedule, and maintain workflows
  • Perform data integration and transformation
  • Troubleshoot issues
  • Document processes and communicate technical concepts clearly
  • Continuously enhance attribution and forecasting engines
  • End-to-end ownership of software objectives, requirements, deliverables, and specifications

Requirements

  • 12+ years experience in Scala programming
  • Extensive experience with Apache Spark for big data processing
  • Experience with on-premises and cloud operations, especially AWS and GCP
  • Hands-on experience with Python
  • In-depth understanding of Hadoop ecosystem including HDFS, YARN, MapReduce
  • Strong SQL skills for handling large datasets
  • Experience building, scheduling, and maintaining DAG workflows
  • Familiarity with data warehousing concepts and technologies
  • Experience with GIT or equivalent source control
  • Understanding of unit and integration test frameworks
  • Ability to collaborate with stakeholders to understand requirements
  • Ability to work under tight deadlines and prioritize tasks
  • Mentoring junior staff
  • Good written and verbal communication skills in English
  • Good understanding of Agile methodologies (SCRUM)

Tech stack

ScalaApache SparkHadoopPythonSQLGITDatabricksDelta LakeELK stackTableauPower BIGrafanaDockerKubernetesAWSGCP

Benefits

Employee well-being focusCollaborative work environmentOpportunities for growth, learning, and career advancementInnovation-driven cultureWork-life balance and flexibilityDiversity and inclusion commitment

Apply now

Ready to take the next step in your career? Click the button below to continue to the application process.

Continue to applicationBrowse more jobs

Similar jobs

More roles worth a look

Related opportunities based on specialty and working model so candidates can keep momentum.