Data Engineer

Key details

Full job description

Seeking a Data Engineer with 6+ years of experience building production-grade data pipelines. Must have hands-on experience with Apache Spark (PySpark) and SQL for large-scale data transformation. Experience with Azure cloud platform and cloud-native data storage such as Data Lake or Blob Storage is required. Familiarity with Databricks and data governance/catalog tools like Unity Catalog is essential. Candidate should understand data governance and access control models (RBAC/ABAC) and have experience working with sensitive, multi-tenant data. Experience integrating data pipelines with BI/visualization tools such as Power BI or Tableau is required. Knowledge of API-based data ingestion tools/connectors like Adverity is a plus. Strong software engineering practices including version control, CI/CD, testing, and code review are expected. Good communication skills and ability to collaborate cross-functionally with product, engineering, and client-facing teams are necessary. Nice to have experience includes workflow orchestration tools (Apache Airflow), identity/access management integrations (Okta, Entra ID), service mesh technologies (Istio), containerized deployments (AKS/Kubernetes), sustainability or carbon accounting data models, and C4 model architecture documentation (PlantUML or similar).