AdTechTalent
Engineering4 months agoHybrid

Epsilon

Lead Software Engineer

pythonscalaapache sparkbig datagoogle bigqueryterraformdatabrickselk stackdockerkubernetesagilescrumdata pipelinesdata engineeringmentoringcloudgcpawssql

Key details

Salary

Not specified

Employment type

Full-time

Seniority

Lead

Years experience

5-10

Location

Bengaluru, India

Full job description

Lead Software Engineer role in Epsilon Product Development team focused on designing, implementing, and optimizing data processing solutions using Google workflows, Scala, Spark, and Hadoop. Responsibilities include deploying big data solutions on on-premises and cloud infrastructure, building and maintaining workflows, data integration and transformation, troubleshooting, documentation, and mentoring junior staff. Requires 7+ years experience in Python/Scala, Apache Spark, Google BigQuery, SQL, Terraform, and GIT. Strong communication skills and Agile SCRUM experience needed. Advantageous experience includes retail media/ad-tech, Databricks, ELK stack, data visualization tools, Docker, Kubernetes, and AWS/GCP certifications. Location: Bengaluru, Karnataka, India.

What you'll do

  • Design, implement, and optimize data processing solutions using Google workflows, Scala, Spark, and Hadoop
  • Collaborate with cross-functional teams to deploy big data solutions on on-premises and cloud infrastructure
  • Build, schedule and maintain workflows
  • Perform data integration and transformation
  • Troubleshoot issues
  • Document processes and communicate technical concepts clearly
  • Continuously enhance attribution and forecasting engines
  • Mentor junior staff

Requirements

  • 7+ years experience in Python/Scala programming
  • Extensive experience with Apache Spark for Big Data processing
  • Experience designing, developing and maintaining scalable on-prem and cloud environments, especially on Google BigQuery
  • Strong SQL skills for RDBMS/Warehouse handling TBs of data
  • Proficiency in performance tuning of BigQuery jobs and optimizing resource usage
  • Hands-on experience with Terraform for GCP infrastructure modules
  • Expertise in designing and implementing scalable, fault-tolerant data pipelines with monitoring and alerting
  • Experience with GIT or equivalent source control
  • Solid understanding of Unit and integration test frameworks
  • Ability to collaborate with stakeholders and work within tight deadlines
  • Mentoring junior staff
  • Good understanding of Agile methodologies – SCRUM
  • Strong written and verbal communication skills in English

Tech stack

PythonScalaApache SparkHadoopGoogle WorkflowsGoogle BigQuerySQLTerraformGITDatabricksDelta LakeELK StackElasticsearchLogstashKibanaTableauPower BIGrafanaDockerKubernetesAWSGCP

Benefits

Opportunities for growth through learning, development and career advancementFlexible work-life balanceInclusive and diverse work environmentCollaborative cultureFocus on employee well-being

Apply now

This MVP uses a placeholder application flow. In production, this section can connect to an external apply URL or a native application form.

Continue to applicationBrowse more jobs

Similar jobs

More roles worth a look

Related opportunities based on specialty and working model so candidates can keep momentum.