Principal Database Infrastructure Engineer

What you'll do

• Design and implement the physical plan distribution pass, including network shuffle, coalesce, and partition isolator insertion
• Own the plan serialization codec that ships sub-plans to workers, and maintain the S3 and Flight result exchange paths
• Own worker discovery and heartbeating, and lead development of the next generation of load balancing: a work-stealing protocol and a replication-aware hash ring
• Work across cost-based join reordering, cross-stage bloom filter cascade, scan deduplication, and selectivity estimation
• Upstream changes to DataFusion fork where appropriate and maintain delta where not
• Own the NVMe LRU cache over S3, Parquet read strategies including full-file and range reads, and Iceberg partition pruning and snapshot handling
• Close the performance gap on queries compared to Snowflake, focusing on multi-shuffle plans and redistribution after scalar-subquery extraction
• Use TPC-H benchmarks as performance scorecard

Requirements

• 8+ years of software engineering experience with significant depth in database infrastructure, distributed systems, or data platform engineering
• Strong systems programming in Rust, or deep C++ or Go experience with a clear path to Rust, including async runtimes such as Tokio and concurrent data structures
• Proven experience building or significantly modifying a distributed data system such as a query engine, stream processor, distributed database, or large-scale data pipeline, with a solid understanding of shuffles, partitioning, and network and memory bottlenecks
• Fluency with columnar formats and vectorized execution, including Arrow, Parquet, and the mechanics behind their performance characteristics
• Strong grounding in distributed systems fundamentals: consistent hashing, leader and heartbeat protocols, backpressure, partial failure, and graceful degradation
• A performance engineering mindset: you profile before optimizing and defend changes with real benchmark numbers
• Direct experience with Apache DataFusion or another SQL query planner or optimizer such as Spark Catalyst, Calcite, Trino, ClickHouse, or DuckDB (strongly preferred)
• Experience with Apache Iceberg or a comparable open table format such as Delta Lake or Hudi (strongly preferred)
• Familiarity with Kubernetes and cloud infrastructure including EKS, S3, and IRSA, particularly on ARM or Graviton (strongly preferred)
• Query optimizer experience including join ordering, predicate pushdown, and cardinality or selectivity estimation (strongly preferred)
• Experience with a distributed SQL store such as CockroachDB, Flight SQL or gRPC, or contributions to open source data infrastructure projects (nice to have)
• Experience building or working with developer tooling in agentic or programmatic data access contexts (nice to have)

Tech stack

Benefits