Staff AI Engineer, Agentic AI Harness & Quality - Moveworks
$176k - $308k • Mountain View, CALIFORNIA, United States • Full-time
Posted 7d ago
Job Location
Mountain View, CALIFORNIA, United States
Tech Stack
Remote Work Policy
On-site
Employment Type
Full-time
Categories
AI Agent Engineer
About the job
We are seeking an experienced software engineer to join our Agentic AI Harness & Quality team, focusing on expanding customer-facing agentic AI capabilities. You will leverage modern agentic AI tools, including frontier LLMs and infrastructure for tuning, evaluating, and serving AI agents in production. This is an eval-driven, data-centric role where you'll collaborate with a world-class annotation team to build high-quality, inclusive, and privacy-preserving datasets. Your work will focus on improving AI performance in production, encompassing accuracy, quality, latency, reliability, and the overall end-to-end user-facing system. We aim to build the highest-performing enterprise assistant platform, and your contributions will be crucial in achieving this goal through close collaboration across teams.
Responsibilities
- Apply software engineering and compound AI system engineering to create lasting customer value.
- Drive quality in agent behavior, increasing success rates and decreasing misbehavior.
- Address challenges in AI harness engineering, including agent cognitive architecture, eval benchmark creation, context engineering, generative UI, dynamic agent orchestration, multimodal I/O, multilinguality, conversational memory management, reasoning strategies, grounding and verifiability, deployment safety, and automatic harness optimization.
- Read, discuss, and build upon the latest AI research and open-source repositories and models.
- Research and develop innovative, scalable, and dynamic solutions to complex problems.
- Utilize machine learning fundamentals and LLMs to improve the agentic harness and its AI components, evaluate them through experiments, and productionize solutions at scale.
- Partner with web and chat platform engineering teams to translate harness investments into user experiences.
Requirements
- Demonstrated ability to ship product improvements with production-quality, fully unit-tested code and rigorously-evaluated updates to models, prompts, context curation code, or other tunable system components.
- Solid grasp of LLM and agent evaluation fundamentals, particularly for agent action trajectories, generated text, and non-uniform sampling regimes.
- Attention to detail and a high standard for data quality in eval datasets.
- Thoughtfulness regarding API contracts and the implications of exposing platform features.
- Proficiency in Python and/or Golang, with experience in a Mac development environment and assistance from coding agents.
- Familiarity with industry-standard agent architectures and large language models.
- Critical reading skills to identify subtle "miscommunication bugs" in agent instructions.
- Ability to fix miscommunication bugs through precise writing and evaluations.
- A proactive approach with a high degree of ownership and a desire to move fast.
- Strong curiosity for engineering outside of immediate disciplines and a commitment to continuous learning in agentic AI.
Benefits
- Equity (when applicable)
- Variable/incentive compensation
- Health plans, including flexible spending accounts
- 401(k) Plan with company match
- ESPP
- Matching donations
- Flexible time away plan
- Family leave programs
