AdTechTalent
Engineering1 month agoHybrid

Epsilon

Lead Databricks Administrator

databricksawsazuregcpterraformsqlunixlinuxpythonscalaci/cdclouddatabase administrationdata platformidentity and access managementrbacssoscimunity catalogdata governanceautomationperformance tuningplatform administration

Key details

Salary

$98K – $182K

Employment type

Full-time

Seniority

Lead

Years experience

5-10

Location

Chicago, United States; Irving, United States

Full job description

Lead Databricks Administrator role responsible for managing enterprise Databricks environments on AWS, Azure, and GCP. Duties include platform administration, enforcing standards, governance, automation, monitoring, troubleshooting, and mentoring. Requires 7+ years in database or data platform administration, 3+ years with Databricks, strong knowledge of cloud platforms, identity and access management, infrastructure automation, and scripting. Must have strong problem-solving and communication skills. Benefits include flexible time off, paid holidays, health coverage, 401(k), and more. Salary range: $98,000 - $182,000 annually. Locations: Chicago, Illinois and Irving, Texas.

What you'll do

  • Lead Databricks platform administration across enterprise workspaces, accounts, and cloud environments
  • Define and enforce standards for workspace provisioning, resource naming, access controls, compute configuration, and operational governance
  • Drive consistent administration practices across Databricks environments to reduce risk, improve supportability, and support long-term platform growth
  • Partner with Product, Engineering, Security, Cloud, and Operations teams to align Databricks platform setup with enterprise security and compliance expectations
  • Oversee workspace setup, cluster configuration, SQL Warehouse administration, job execution standards, and platform lifecycle management
  • Support governance and access control processes, including least privilege access, group management, service principal usage, and admin access reviews
  • Lead automation efforts for repeatable provisioning, configuration management, reporting, and operational controls
  • Monitor platform health, usage, performance, and cost trends; identify opportunities for optimization and improvement
  • Troubleshoot complex issues across Databricks, cloud infrastructure, identity, networking, data access, and workload execution
  • Evaluate new Databricks capabilities and recommend adoption where they improve governance, scalability, security, or operational efficiency
  • Maintain platform documentation, standards, runbooks, and operational procedures
  • Mentor administrators and engineers, promote best practices, and help improve the maturity of Databricks platform operations
  • Support production operations, incident response, and on-call rotation as needed

Requirements

  • 7+ years of experience in database administration, data platform administration, cloud infrastructure, or related technology roles
  • 3+ years of hands-on experience administering Databricks workspaces, accounts, clusters, jobs, SQL Warehouses, and related platform services
  • Strong understanding of Databricks workspace management, account administration, compute configuration, access controls, platform governance, and operational best practices
  • Experience supporting Databricks environments across one or more major cloud platforms such as AWS, Azure, or GCP
  • Hands-on experience with identity and access management concepts, including SSO, SCIM provisioning, RBAC, service principals, groups, and least privilege access models
  • Working knowledge of Unity Catalog, data governance, secure data access patterns, catalog-level permissions, and data access auditing
  • Experience implementing and enforcing platform standards such as workspace naming, group naming, resource naming, admin access controls, provisioning workflows, and configuration baselines
  • Experience maintaining platform hygiene, including workspace reviews, admin access reviews, group cleanup, unused resource identification, and standards compliance
  • Experience with infrastructure automation and configuration management using Terraform, scripting, APIs, CI/CD pipelines, or similar tools
  • Ability to monitor, troubleshoot, and optimize Databricks workloads for performance, reliability, scalability, and cost efficiency
  • Strong understanding of Databricks compute patterns, cluster policies, job execution behavior, SQL Warehouse usage, and workload isolation practices
  • Ability to diagnose issues across Databricks, cloud infrastructure, networking, identity, storage access, permissions, and data pipeline execution
  • Strong SQL, Unix/Linux, and scripting skills
  • Python or Scala experience is a plus
  • Experience in creating and maintaining operational documentation, runbooks, standards, procedures, and support guides
  • Experience working with vendor support, internal engineering teams, and security teams to resolve issues within expected timelines
  • Strong understanding of change management, SDLC practices, incident response, release coordination, and production support expectations
  • Ability to work independently with minimal supervision while managing multiple priorities across client and platform initiatives
  • Excellent written and verbal communication skills
  • BS in Computer Science or related technical discipline, or equivalent practical experience

Tech stack

DatabricksAWSAzureGCPTerraformSQLUnix/LinuxPythonScalaCI/CDSSOSCIMRBACUnity Catalog

Benefits

Flexible time off (FTO)15 paid holidaysPaid sick timeParental/new child leaveChildcare & elder care assistanceAdoption assistanceComprehensive health coverage401(k)Tuition assistanceCommuter benefitsProfessional developmentEmployee recognitionCharitable donation matchingHealth coaching and counseling

Apply now

Ready to take the next step in your career? Click the button below to continue to the application process.

Similar jobs

More roles worth a look

Related opportunities based on specialty and working model so candidates can keep momentum.