AdTechTalent
Data Science1 month agoHybrid

Epsilon

Senior Software Engineer

databrickshadoopsparkpysparkhivesqldelta lakeawsdata engineeringetlperformance tuningbig datadata pipelinesjiragitbitbucket

Key details

Salary

Not specified

Employment type

Full-time

Seniority

Mid-level

Years experience

5-10

Location

Bengaluru, India

Full job description

The Data Engineering team at Epsilon is seeking a mid-level Data Engineer with 5-8 years of experience to develop, unit test, and implement data applications primarily in the Hadoop ecosystem and Databricks. The role involves designing, building, and optimizing scalable data pipelines and ETL frameworks using Spark, PySpark, Hive, SQL, and Delta Lake. Candidates must have hands-on experience with Databricks, AWS, performance tuning for large datasets, and tools like JIRA and GIT/Bitbucket. Responsibilities include collaborating with technical leads and architects, contributing to data modeling and governance, troubleshooting production issues, and improving deployment processes. The position is based in Bengaluru, Karnataka, India.

What you'll do

  • Design, develop, and maintain data pipelines and ETL frameworks using Spark, PySpark, Hive, and SQL
  • Design and develop scalable data pipelines and processing frameworks on Databricks using PySpark, SQL, and Delta Lake
  • Develop efficient, reusable, and reliable code for data processing and transformation
  • Optimize and tune Spark jobs for performance and scalability
  • Work with Technical Leads, Architects and data platform teams to implement robust data solutions
  • Contribute to data modeling, quality, and governance initiatives
  • Perform detailed analysis, troubleshooting, and root cause analysis for production issues
  • Participate in code reviews and enforce best coding and design practices
  • Collaborate with cross-functional teams to deliver high-quality software solutions
  • Improve and optimize deployment challenges and help deliver reliable solutions
  • Interact with technical leads and architects to solve challenges faced by Data Engineering teams
  • Contribute to continuous improvement of development and delivery processes

Requirements

  • BE / B.Tech / MCA – No correspondence course
  • 5-8 years of experience
  • Hands-on experience in building and optimizing data solutions on the Hadoop ecosystem leveraging PySpark
  • Good knowledge and experience with Databricks
  • Experience working with AWS
  • Experience with performance tuning for large data sets
  • Experience with JIRA for user-story/bug tracking
  • Experience with GIT/Bitbucket

Tech stack

HadoopDatabricksSparkPySparkHiveSQLDelta LakeAWSJIRAGITBitbucket

Benefits

Opportunity to design, build, and optimize large-scale data solutionsExposure to diverse data engineering challengesHands-on experience with modern data platforms, cloud ecosystems, and automation frameworksCollaborative and agile work environmentBeing part of a global Data team impacting data-driven decision-makingEndless opportunities for growth through learning, development and career advancementFocus on employee well-beingWork-life balance and flexibilityDiversity and inclusion commitment

Apply now

This MVP uses a placeholder application flow. In production, this section can connect to an external apply URL or a native application form.

Continue to applicationBrowse more jobs

Similar jobs

More roles worth a look

Related opportunities based on specialty and working model so candidates can keep momentum.