Data Engineer

azuredatabrickssqlpythonpysparksparketleltdata engineeringazure data factoryazure synapseadlsci/cdterraformarm templatesazure devopsgithub actionsmonitoringdata pipelinesdata lakesstreamingevent hubskafkapower bidelta lakelakehousedata modeling

Key details

Salary

Not specified

Employment type

Full-time

Seniority

Senior

Years experience

3-5

Location

Pune, India

Full job description

Looking for a Senior Data Engineer with 3 to 7 years experience to build, optimize, and maintain scalable cloud data platforms using Microsoft Azure and Databricks. Role involves developing reliable data pipelines and analytics-ready datasets. Requires strong hands-on expertise in SQL and Python, building batch and basic streaming data pipelines in cloud. Must have experience with Azure Data Lake Storage Gen2, Azure Data Factory, Azure Databricks, Azure Synapse Analytics, and cloud-native data lakes. Strong skills in SQL, Python, PySpark, Spark, data modeling, transformations, and query performance tuning required. Knowledge of Infrastructure as Code (ARM Templates/Terraform), CI/CD pipelines (Azure DevOps/GitHub Actions), and Azure monitoring tools needed. Responsibilities include building ETL/ELT pipelines, developing SQL and Python/PySpark transformations, ingesting and processing data, supporting pipeline deployment and monitoring, troubleshooting failures, and collaborating with architects and product teams. Education: Bachelor's or Master's degree or equivalent. Certifications preferred: Microsoft Certified Fabric Data Engineer Associate, Azure Solutions Architect Expert, Snowflake Core. Shift timing: 12 PM to 9 PM and/or 2 PM to 11 PM IST.

What you'll do

Build scalable ETL/ELT pipelines using Azure and Databricks
Develop SQL-based transformations and Python/PySpark pipelines
Ingest and process data using ADF, Databricks, and ADLS
Build analytics-ready data models optimized for performance and cost
Support deployment and execution of data pipelines across environments
Monitor data pipeline health, performance, and data quality
Troubleshoot pipeline failures and perform root-cause analysis
Follow Azure security, reliability, and scalability best practices
Work closely with architects and product teams to understand requirements
Translate business and analytics needs into working Azure data solutions
Contribute to documentation, code reviews, and engineering standards

Requirements

Hands-on experience delivering Azure-based data engineering solutions
Strong hands-on experience with Azure data services including ADLS Gen2, ADF, Databricks, Synapse Analytics
Experience designing cloud-native data lakes and analytical data stores
Solid understanding of batch data pipelines and basic streaming concepts
Strong SQL skills including complex queries, joins, aggregations, and transformations
Strong Python skills for ETL/ELT and data engineering use cases
Hands-on experience with PySpark in Databricks
Strong understanding of data modeling, transformations, and query performance tuning
Hands-on experience with Spark/PySpark on Databricks
Experience handling structured and semi-structured data
Understanding of partitioning, schema evolution, and data validation concepts
Working knowledge of Infrastructure as Code (ARM Templates and/or Terraform)
Basic experience with CI/CD pipelines (Azure DevOps or GitHub Actions)
Understanding of logging and monitoring using Azure Monitor / Log Analytics
Ability to work effectively with architects, QA, DevOps, and business stakeholders
Good communication skills

Tech stack

Azure Data Lake Storage Gen2Azure Data FactoryAzure DatabricksAzure Synapse AnalyticsSQLPythonPySparkSparkARM TemplatesTerraformAzure DevOpsGitHub ActionsAzure MonitorLog AnalyticsAzure Event HubsKafkaPower BIDelta LakeLakehouse architecture

Apply now

This MVP uses a placeholder application flow. In production, this section can connect to an external apply URL or a native application form.

Continue to application Browse more jobs

Company

Merkle

Merkle, a dentsu company, powers the experience economy. For more than 35 years, the company has put people at the heart of its approach to digital business transformation. As the only integrated experience consultancy in the world with a heritage in data science and business performance, Merkle delivers holistic, end-to-end experiences that drive growth, engagement, and loyalty. Merkle’s expertise has earned recognition as a “Leader” by top industry analyst firms, in categories such as digital transformation and commerce, experience design, engineering and technology integration, digital marketing, data science, CRM and loyalty, and customer data management. With more than 16,000 employees, Merkle operates in 30+ countries throughout the Americas, EMEA, and APAC.

Industry

Business Consulting and Services

Company size

10,001+

Website

https://www.merkle.com

Posted

4 days ago

Category: Data Science

Similar jobs