Role Summary Senior subject-matter expert for the operations, observability, and lifecycle management of AI agents in production ("AgentOps” / LLMOps). Owns the frameworks and practices to safely deploy, monitor, evaluate, and continuously improve live agents — ensuring reliability, safety, cost-efficiency, and business-KPI performance across the Client Agent Factory. Key Responsibilities
Define and operate the AgenticOps framework: agent registry, versioning, guarded rollout, and rollback for production agents.
Establish continuous evaluation and monitoring: quality, autonomy, safety (guardrails, Model Armor), latency, cost, and reuse metrics.
Implement observability and tracing for multi-agent systems (Agent Engine Observability, Cloud Monitoring/Logging/Trace). Own the 5-gate validation-to-production process and post-release escape management for delivered agents.
Design human-in-the-loop (HITL) supervision, feedback loops, and automated pre-production simulations for safe rollout.
Track and report agent business KPIs (CSAT, TAT, MTTR, cost savings) via AgentScore / Agent 360 dashboards.
Drive cost governance for agent runtimes: model tiering, context caching, batch/flex inference, budget caps and alerts.
Collaborate with DevOps SME (deploy) and AI & Data SME (grounding) to close the build-deploy-operate-improve loop; advise Client on AgenticOps ownership transfer.
Mandatory (Must-Have) Skills
Strong LLMOps / MLOps / AgentOps experience operating GenAI or agentic systems in production.
Hands-on with Google Cloud agent runtimes: Vertex AI, Agent Engine, and observability tooling.
Agent evaluation and safety: eval frameworks, guardrails, Model Armor, HITL, prompt/robustness testing.
Monitoring, tracing, and reliability engineering (SRE) for AI workloads.
Cost governance and performance tuning for LLM/agent workloads.
Proficiency in Python; strong grasp of agent lifecycle and governance.
Preferred (Good-to-Have) Skills
Experience with ADK, A2A, MCP, and multi-agent orchestration in production.
BigQuery/Looker for agent analytics and KPI dashboards.
Responsible-AI, model governance, and audit/compliance frameworks.
Prior enterprise-scale AI platform operations experience.
Experience & Certifications
9–12+ years in ML/AI platform operations, SRE, or LLMOps with production agentic/GenAI exposure (Tier 5–6).
Google Cloud Professional (ML/DevOps) certification preferred.
Numbers & Facts
Location
Santa Clara, CA
Industry
Other/Not Classified
Company Size
100 to 499 employees
Skills
Artificial Intelligence (AI)unmatched
Artificial Intelligence (AI) Agentsunmatched
Budgetingunmatched
Cachingunmatched
Cloud Computingunmatched
Continuous Improvementunmatched
DevOpsunmatched
Lookerunmatched
MCP - Microsoft Certified Professionalunmatched
Machine Toolunmatched
Metricsunmatched
Performance Metricsunmatched
Performance Tuning/Optimizationunmatched
Production Systemsunmatched
Python Programming/Scripting Languageunmatched
Quality Monitoringunmatched
Release Management/Engineeringunmatched
Reliability Engineeringunmatched
Reporting Dashboardsunmatched
Safety/Work Safetyunmatched
🎯
Be found by employers
5,500+ employers search our resume database daily. Add yours to get found by recruiters looking for candidates like you.
Level up your application
Professional resume templates
Browse dozens of recruiter approved resume templates, layouts and formats. Choose your favorite and make it your own in minutes.