Data Engineer - Samba Platform

What you'll do

• Build scalable data product architecture, capable of supporting both internal and external data consumers
• Responsible for modernizing our data frameworks and integrations with Databricks and BigQuery
• Upgrade and reduce toil for developers on Apache Airflow
• Develop and optimize data transformations using Apache Spark (PySpark/Scala)
• Build procedures and guidelines to help teams operate with data
• Identify bottlenecks in our development lifecycle and find solutions to improve them
• Drive innovation throughout the tech org by evangelizing and educating teams on best practices and new technologies
• Work directly with our data teams and FinOps teams to drive efforts that span across teams
• Implement data governance, access control, and auditing using Databricks Unity Catalog
• Build and integrate automated, reusable data validation suites using data quality frameworks (Great Expectations or similar)
• Implement monitoring and anomaly detection systems for data quality, reliability and performance
• Develop and manage REST APIs to support secure data access, automation, and integration
• Collaborate with data scientists, analysts, and software engineers to deliver governed, reusable data assets
• Implement monitoring, logging, and alerting for data workflows
• Optimize cost and performance of cloud-based data infrastructure

Requirements

• Bachelor’s degree in Computer Science, Engineering, or a related field (or equivalent experience)
• 5+ years of experience in data engineering or a related role
• Strong hands-on experience with Databricks, Apache Spark and BigQuery or Snowflake
• Proven experience with Modern table formats such as Delta Lake and Iceberg
• A deep understanding of the data lifecycle and how teams operate with data
• Hands-on experience implementing data governance and metadata management using Databricks Unity Catalog
• Experience managing and extending Apache Airflow (custom operators, plugins, infrastructure)
• Experience with Kubernetes
• Solid experience with AWS cloud services, especially S3 and data-related services
• Experience with data validation and data quality principles and working with SLA systems
• Proficiency in Python and SQL
• Experience with data modeling, data lakes, and lakehouse architectures and a strong understanding of distributed systems and big data processing
• Preferred Qualifications: Experience with Infrastructure as Code (Terraform)
• Knowledge of CI/CD pipelines for data platforms
• Experience working in Agile/Scrum environments
• AWS, Databricks, or cloud data certifications
• Experience working with data clean room environments, including implementation or collaboration within Snowflake, Databricks, or BigQuery ecosystems
• Experience with streaming data pipelines (Kafka, Flink)

Tech stack