New: upload your resume and get matched to AI jobs. Free to start →

Staff AI Engineer, Agentic AI Harness & Quality - Moveworks

$176k - $308k • Mountain View, CALIFORNIA, United States • Full-time

Posted 7d ago

Job Location

Mountain View, CALIFORNIA, United States

Tech Stack

Remote Work Policy

On-site

Employment Type

Full-time

Categories

AI Agent Engineer

About the job

We are seeking an experienced software engineer to join our Agentic AI Harness & Quality team, focusing on expanding customer-facing agentic AI capabilities. You will leverage modern agentic AI tools, including frontier LLMs and infrastructure for tuning, evaluating, and serving AI agents in production. This is an eval-driven, data-centric role where you'll collaborate with a world-class annotation team to build high-quality, inclusive, and privacy-preserving datasets. Your work will focus on improving AI performance in production, encompassing accuracy, quality, latency, reliability, and the overall end-to-end user-facing system. We aim to build the highest-performing enterprise assistant platform, and your contributions will be crucial in achieving this goal through close collaboration across teams.

Responsibilities

  • Apply software engineering and compound AI system engineering to create lasting customer value.
  • Drive quality in agent behavior, increasing success rates and decreasing misbehavior.
  • Address challenges in AI harness engineering, including agent cognitive architecture, eval benchmark creation, context engineering, generative UI, dynamic agent orchestration, multimodal I/O, multilinguality, conversational memory management, reasoning strategies, grounding and verifiability, deployment safety, and automatic harness optimization.
  • Read, discuss, and build upon the latest AI research and open-source repositories and models.
  • Research and develop innovative, scalable, and dynamic solutions to complex problems.
  • Utilize machine learning fundamentals and LLMs to improve the agentic harness and its AI components, evaluate them through experiments, and productionize solutions at scale.
  • Partner with web and chat platform engineering teams to translate harness investments into user experiences.

Requirements

  • Demonstrated ability to ship product improvements with production-quality, fully unit-tested code and rigorously-evaluated updates to models, prompts, context curation code, or other tunable system components.
  • Solid grasp of LLM and agent evaluation fundamentals, particularly for agent action trajectories, generated text, and non-uniform sampling regimes.
  • Attention to detail and a high standard for data quality in eval datasets.
  • Thoughtfulness regarding API contracts and the implications of exposing platform features.
  • Proficiency in Python and/or Golang, with experience in a Mac development environment and assistance from coding agents.
  • Familiarity with industry-standard agent architectures and large language models.
  • Critical reading skills to identify subtle "miscommunication bugs" in agent instructions.
  • Ability to fix miscommunication bugs through precise writing and evaluations.
  • A proactive approach with a high degree of ownership and a desire to move fast.
  • Strong curiosity for engineering outside of immediate disciplines and a commitment to continuous learning in agentic AI.

Benefits

  • Equity (when applicable)
  • Variable/incentive compensation
  • Health plans, including flexible spending accounts
  • 401(k) Plan with company match
  • ESPP
  • Matching donations
  • Flexible time away plan
  • Family leave programs