Lead development and management of architecture for chaining LLM agents, tools, models, and workflows. Develop shared Context Graph for persistent awareness across agents and products. Implement context streaming services for real-time user and system data. Define telemetry frameworks for agent operation analysis and continuous improvement. Build evaluation frameworks for agent quality and workflow completion. Create Model Workbench for non-technical users to leverage AI models and workflows. Oversee registration and governance of Model Context Protocol servers and platform capabilities. Align cross-functional teams on agentic architecture, standards, and priorities. Define platform standards and governance. Troubleshoot and prototype solutions using LangSmith and observability tools. Promote rapid prototyping and evidence-based iteration. Requires experience leading complex AI and LLM-based products, strong understanding of agentic systems, context and knowledge architectures, telemetry, organizational alignment, systems thinking, technical proficiency with AI tools, problem-solving, and communication skills.
What you'll do
Develop and manage architecture for chaining LLM agents, tools, models, and workflows across complex use cases
Similar jobs
More roles worth a look
Related opportunities based on specialty and working model so candidates can keep momentum.
Bellevue, United States; Boston, United States; Boulder, United States; Chicago, United States; Denver, United States; Los Angeles, United States; New York, US; San Francisco, United States; San Jose, United States; Seattle, United States; Ventura, United States•14 days ago
Lead development of a shared Context Graph for persistent awareness of users, brands, accounts, workflows, capabilities, data, prior actions, goals, and outcomes
Implement and manage context streaming services for real-time awareness of user actions, application state, system events, and business data
Define telemetry framework to understand agent operation in production and build feedback loops for continuous improvement
Build evaluation frameworks for testing agent quality, reliability, routing, context utilization, tool execution, and workflow completion
Lead creation of a Model Workbench for non-technical users to leverage LLMs, ML, agents, and workflows safely
Oversee registration, documentation, governance, and discoverability of Model Context Protocol servers, tools, agents, and platform capabilities
Drive alignment across Product, Engineering, Data Science, Design, Analytics, Security, and business stakeholders on agentic architecture and platform priorities
Define platform standards and governance for agents, tools, context sources, telemetry, and workflows
Participate in troubleshooting and debugging using LangSmith and observability platforms
Promote culture of rapid prototyping, experimentation, and evidence-based iteration
Requirements
Experience leading complex technical products involving LLMs, AI agents, workflow systems, developer platforms, ML infrastructure, or AI-driven applications
Strong understanding of LLM agents, tool use, orchestration, multi-agent workflows, state management, context management, and architectural patterns for agentic systems at scale
Experience designing or working with context graphs, knowledge graphs, semantic systems, memory architectures, metadata platforms, or similar systems
Strong understanding of instrumentation, telemetry, evaluation, and observability for complex software or AI systems
Ability to align senior stakeholders and cross-functional teams around shared technical architecture, product priorities, ownership boundaries, and operating standards
Systems thinking across product experience, model behavior, data, infrastructure, APIs, organizational ownership, and operational processes
Strong technical background with hands-on familiarity with LangSmith, APIs, workflow orchestration, LLM agent chaining, MCP, evaluation frameworks, and AI development environments
Ability to troubleshoot ambiguous technical problems, rapidly prototype solutions, and translate experimentation into scalable decisions
Excellent communication skills to translate technical concepts into clear product strategies and decisions for technical and non-technical audiences
