
Posted 2 days ago
Staff Engineer, Agentic AI
AI Summary
Leads technical development of agentic AI systems that help engineers complete complex workflows in desktop engineering software, owning agent architecture, evaluation infrastructure, and cost metrics while leading a small team with hands-on coding.
About this role
About the Role
Lead the development of agent intelligence that helps engineers complete complex workflows across desktop engineering software. You will be a hands-on technical lead for a small AI team, partnering with user research and domain experts to deliver reliable, cost-efficient capabilities grounded in real engineering workflows.
What You'll Do
Own agent architecture for tool calling, state management, error recovery, model routing, and context management.
Define and improve task success metrics, establishing baselines and tracking workflow completion, cost, and failure modes.
Build rigorous, reproducible evaluation infrastructure that translates validated user stories into test cases.
Work with researchers, domain experts, and engineers to map user workflows and prioritize valuable agent capabilities.
Set per-task token budgets and measure cost per completed workflow.
Lead a small team through technical direction, design reviews, code reviews, and hands-on implementation.
Collaborate with product, integrations, and customers to align agent behavior with real-world use.
What We're Looking For
At least 7 years of software engineering experience, including 2 or more years building LLM agents that take real-world actions.
Deep knowledge of LLM application architecture, including model selection, context management, retrieval, tool calling, and orchestration.
Experience creating agent evaluation and benchmarking frameworks for task completion, cost efficiency, and failure analysis.
Strong Python skills and familiarity with function calling, tool APIs, observability or tracing, and evaluation tools.
Hands-on technical leadership experience guiding a small engineering team while continuing to write production code.
Experience shipping AI or LLM tools over proprietary engineering data or desktop engineering software.
Experience with desktop automation or application APIs, such as COM, and familiarity with enterprise workstation constraints.
Background in mechanical engineering software, CAD/CAE, PLM, or a related engineering domain.
Compensation & Benefits
Base salary: $160,000 to $250,000 annually. Visa sponsorship is not available.
Location
On-site in San Francisco, California, United States.
Skills
Explore related jobs
More jobs at Clera
Similar Agent Evaluation jobs
Jobs in San Francisco
- S
Teaching Artist (on-call)SFJAZZ · San Francisco, California - C
Assistant Community ManagerCareers Related · San Francisco, California - M
Asset Protection Captain, San Francisco Union Square - Full TimeMacy's · San Francisco, CA - E
Associate Director of Food ServiceEVERY™ · San Francisco, California - S
Senior Communications Manager, SponsorshipsSubstack · San Francisco (HQ) - TSocial Media ManagerTuring · San Francisco, California
Browse these categories
Market data for this role
All reports →- SeriesRole reportsOne role family at a time: how many openings, what changed this week, who is hiring, what it pays.
- SeriesSalary reportsWhat employers publish in job postings, by level and workplace. Not self-reported pay.
- Market overviewState of tech hiring, September 2026: up 4.8%Tech hiring rose 4.8% month over month in September 2026, with 411,122 new listings. Customer support and account executive roles led the growth.