Staff Quality Engineer

What you'll do

• Own and drive the end-to-end quality strategy for the Automation and Intelligence Pillar pod's AI products
• Design and build evaluation frameworks for non-deterministic systems including LLM output quality, agent behaviour, regression detection, and guardrail/safety testing
• Develop reliable, scalable automated test suites and tooling for functional, system, API, integration, UI, and regression testing integrated into CI/CD pipelines
• Establish quality metrics and benchmarks such as accuracy, latency, cost, hallucination/failure rates
• Partner with Engineering Managers, Product Managers, Designers, ML/Data Science, and other engineering teams to align requirements and priorities
• Act as a technical mentor and promote QE best practices across the organization
• Participate in technical planning, code review, and architectural decision-making advocating for testability and reliability
• Provide timely, actionable test result feedback to relevant stakeholders

Requirements

• 7+ years of professional, hands-on software quality and test automation experience
• Experience setting technical direction on complex systems
• Deep, hands-on experience with API & UI test automation frameworks such as Playwright, Cypress, Jest, or Junit
• Fluency in at least one of TypeScript, JavaScript, Python, or Java with preference for TypeScript
• Experience testing AI/agentic systems (evals, prompt/regression testing, output validation)
• Experience using AI to scale automation (test generation, self-healing tests, failure triage) with technologies such as OpenAI, Anthropic
• Experience with CI/CD tools like Jenkins, Github Actions, or CircleCI
• Strong understanding of scalable system design for web-based APIs and backend processing systems
• In-depth understanding of QA methodologies in a software company with fast release cycles
• Ability to define test case design standards and coverage strategy including edge/negative, compatibility, performance and concurrency scenarios
• Experience owning a test case management practice (e.g. TestRail)
• Knowledge of React, including adding data-testids
• Ability to explore a codebase and use technical tools to understand technical implementation
• Experience coaching and influencing cross-functional teams on tooling, testing plans and strategies

Tech stack

Benefits