Staff AI Engineer, Agentic AI Harness & Quality - Moveworks
Remote • Mountain View, CALIFORNIA, United States • Full-time
Posted 6h ago
About the job
We are seeking an experienced software engineer to join our Agentic AI Harness & Quality team, focusing on expanding customer-facing agentic AI capabilities. You will leverage modern agentic AI tools, including frontier LLMs and infrastructure for tuning, evaluating, and serving AI agents in production. This is an eval-driven, data-centric role where you'll contribute to building error-free, inclusive, and privacy-preserving datasets. Your work will focus on enhancing AI performance in production, encompassing accuracy, quality, latency, reliability, and the overall user-facing system. We aim to build the world's highest-performing enterprise assistant platform, and your contributions will be crucial to this objective.
Responsibilities
- Apply software engineering and compound AI system engineering to create lasting customer value.
- Drive quality in agent behavior, focusing on goal achievement and minimizing misbehavior.
- Address complex AI harness engineering challenges, including agent cognitive architecture, eval benchmark creation, context engineering, generative UI, dynamic agent orchestration, multimodal I/O, multilinguality, conversational memory management, reasoning strategies, grounding and verifiability, deployment safety, and automatic harness optimization.
- Read, discuss, and build upon the latest AI research and open-source repositories and models.
- Research and develop innovative, scalable, and dynamic solutions to challenging problems.
- Utilize machine learning fundamentals and LLMs to improve the agentic harness and its AI components, evaluate them through experiments, and productionize solutions at scale.
- Partner with web and chat platform engineering teams to translate harness investments into exceptional user experiences.
Requirements
- Demonstrated drive to ship product improvements with production-quality, fully unit-tested code and rigorously-evaluated updates.
- Solid understanding of LLM and agent evaluation fundamentals, particularly for agent action trajectories, generated text, and non-uniform sampling regimes.
- High attention to detail and a strong standard for data quality in eval datasets.
- Thoughtfulness regarding API contracts and the implications of exposing platform features.
- Readiness to work in a Mac development environment, programming in Python and/or Golang with the assistance of coding agents.
- Familiarity with industry-standard agent architectures and large language models.
- Critical reading skills to identify subtle communication issues in agent instructions.
- Ability to resolve communication bugs through precise writing and evaluations.
- Desire to move fast with a high degree of ownership.
- Strong appetite for continuous incremental wins and completing challenging projects rapidly.
- High curiosity for engineering outside of immediate discipline and a desire to stay at the cutting edge of agentic AI.