Sr. Data Engineer

What you'll do

• Design, build, and maintain Snowflake-based pipelines for consumer and household identity assets
• Write complex SQL and Python to transform, deduplicate, and enrich identity data at scale including PII fields
• Investigate data anomalies and quality issues at record level, trace match decisions to source signals
• Build and maintain data models representing consumer and household identity linkage across multiple sources
• Partner with senior engineers and data scientists to enhance AI-assisted matching engine
• Contribute to feature design, scoring logic, model evaluation, and threshold tuning
• Implement and test matching algorithm improvements (AI-driven and rule-based) and measure impact
• Build evaluation tooling such as ground-truth comparisons, match quality dashboards, regression detection
• Drive evolution of matching pipeline towards AI-augmented identity resolution using AI tools daily
• Work cross-functionally with Data Science, Product, and engineering teams to translate identity requirements
• Participate in code reviews and architectural discussions, apply engineering best practices across delivery lifecycle
• Document data models, pipeline logic, and algorithm decisions for technical and non-technical audiences
• Support QA processes and on-call responsibilities for production identity asset pipelines
• Build automated validation frameworks and quality tracking pipelines to monitor asset health and report results

Requirements

• 4+ years of data engineering or software engineering experience with a focus on data-intensive systems
• Strong Python skills with clean, well-structured code
• Deep Snowflake fluency including data modeling, complex querying, Streams and Tasks, performance tuning, preferably Snowpark for Python-native workloads
• Strong SQL fundamentals and experience working with large, messy, real-world datasets
• Experience or curiosity around identity matching, deduplication, record linkage, or data quality at scale
• Comfort working with PII-class data responsibly with awareness of data governance and privacy best practices
• Familiarity with version control (Git), Agile delivery, and CI/CD pipelines
• Comfort applying AI tools in day-to-day engineering work including prompt engineering, LLM-assisted data processing, and AI-augmented pipeline logic
• Nice to have: hands-on exposure to matching algorithms (deterministic, probabilistic, ML/AI-based) and experience evaluating or tuning their performance
• Nice to have: experience building agentic workflows and working with MCP servers
• Nice to have: some Java experience and comfort with JVM-based tooling
• Nice to have: familiarity with consumer or household identity signals (name, address, email, phone, cross-source linkage)
• Nice to have: cloud experience preferably AWS, also Azure or GCP
• Nice to have: Unix/Bash scripting comfort

Tech stack

Benefits