javapythonsparktypescriptjavascriptangularawsrest apimicroserviceskubernetessqlnosqlpysparkapache sparkterraformansiblejenkinsgithubgitlabbitbucketgocdkafkasnssqsci/cddevopsgenerative aillmsragagentic aidata platformcloud-nativedistributed systemsdata warehousingdata lakesdelta lake
Key details
Salary
$113K – $209K
Employment type
Full-time
Seniority
Lead
Years experience
5-10
Location
Wakefield, United States
Full job description
Lead Software Engineer role in the CDP team responsible for design, delivery, technical strategy, and execution of large-scale cloud-native data platforms primarily on AWS. Requires 8-10 years of software engineering experience with expertise in Java, Python, Spark, Angular, REST APIs, microservices, Kubernetes, SQL/NoSQL databases, and AWS services. Responsibilities include leading technical initiatives, partnering with stakeholders, owning the software development lifecycle, and mentoring junior engineers. Experience with data warehousing, data lakes, Delta Lake, PySpark, Apache Spark, Infrastructure as Code tools, CI/CD, DevOps, and generative AI technologies is required. Benefits include flexible time off, paid holidays, health coverage, 401(k), parental leave, tuition assistance, and more. Location: Wakefield, Massachusetts, United States. Salary range: USD $112,700 - $209,300 annually.
What you'll do
Deliver large-scale cloud-native data platforms primarily on AWS using REST APIs, microservices, and event-driven applications
Similar jobs
More roles worth a look
Related opportunities based on specialty and working model so candidates can keep momentum.
Work hands-on across the technology stack including Java, Python, Spark, TypeScript, JavaScript, Angular, AWS services, event-driven architectures, and SQL/NoSQL databases
Lead product-wide technical initiatives focused on performance optimization, scalability, reliability, security, governance, and cost efficiency
Partner closely with global engineering, product management, architecture, and business stakeholders to align technical solutions with strategic business objectives
Own the end-to-end software development lifecycle including requirements gathering, solution design, development, deployment, observability, and documentation
Mentor and guide junior engineers fostering a culture of innovation, accountability, collaboration, and technical excellence
Requirements
B.E./B.Tech/M.Tech/MCA in Computer Science, Information Technology, or related field
8-10 years of strong software engineering experience
Experience building scalable UX driven applications and distributed systems architecture
Proven experience designing and building scalable REST APIs, microservices, Kubernetes, and distributed systems
Experience in Data Warehousing, Data Lakes, Delta Lake architecture, and modern big data ecosystem designs
Strong hands-on expertise in Python, Java, Angular
Well-versed with Object oriented design patterns and Functional programming
Expertise in PySpark and Apache Spark
Experience with Kubernetes containerization for eventing and serving
Extensive experience with AWS services such as S3, Lambda, API Gateway, and EventBridge
Strong experience with messaging and event-driven technologies such as Kafka, SNS, and SQS
Solid expertise in relational and NoSQL databases including PostgreSQL, SQL Server, Aurora, DynamoDB, MongoDB, and Redis
Hands-on experience with Infrastructure as Code (IaC) tools such as Terraform and Ansible
Strong understanding of CI/CD and DevOps practices using Jenkins, GitHub/GitLab, Bitbucket, GoCD
Experience implementing robust testing strategies including unit, integration, and regression testing
Strong critical thinking and analytical skills
Exposure to Generative AI technologies including LLMs, RAG architectures, and Agentic AI systems
Nice to have: Working knowledge of PySpark with Databricks
Nice to have: Experience working with Azure and/or Google Cloud Platform (GCP)
Nice to have: Experience building data platforms in privacy-safe or Customer Data Platforms and Marketing Technology environments
Tech stack
JavaPythonSparkTypeScriptJavaScriptAngularAWS (S3, Lambda, API Gateway, EventBridge)REST APIsMicroservicesKubernetesSQLNoSQLPostgreSQLSQL ServerAuroraDynamoDBMongoDBRedisPySparkApache SparkTerraformAnsibleJenkinsGitHubGitLabBitbucketGoCDKafkaSNSSQSCI/CDDevOpsGenerative AILLMsRAG architecturesAgentic AI
Benefits
Flexible time off (FTO)15 paid holidaysPaid sick timeParental/new child leaveChildcare & elder care assistanceAdoption assistanceComprehensive health coverage401(k)Tuition assistanceCommuter benefitsProfessional developmentEmployee recognitionCharitable donation matchingHealth coaching and counseling
Apply now
Ready to take the next step in your career? Click the button below to continue to the application process.
