SDE 2

Key details

Full job description

Role: SDE II - Data Platform in Lucknow. Build and maintain scalable batch and streaming data pipelines using Spark and Flink. Develop integrations with Airflow, Iceberg, and cloud platforms. Apply data modeling, partitioning, and query optimization best practices. Support Kubernetes workloads and troubleshoot issues. Optimize pipelines for performance, cost, and reliability. Implement monitoring, logging, and data quality checks. Participate in on-call and incident resolution. Requirements: 3-6 years in data/platform engineering, strong SQL and data modeling skills, hands-on Spark (PySpark/Scala), exposure to Flink, experience with Airflow or similar tools, familiarity with data lake formats (Iceberg/Delta/Hudi), knowledge of Kubernetes and cloud platforms (GCP/AWS/Azure), programming in Python (required) and Scala (preferred), exposure to CI/CD, Git, and IaC tools (Terraform/Helm). Good to have: Kafka, DBT, Great Expectations, observability tools (Prometheus, Grafana), metadata/catalog tools (OpenMetadata, DataHub). Benefits include continuous learning and career progression programs, equal employment opportunity, and accommodations for disabilities.