Engineering Manager, Agent Oversight
$248k - $311k • Remote • San Francisco, CA; New York, NY
Posted 5d ago
Job Location
San Francisco, CA; New York, NY
Tech Stack
Remote Work Policy
Fully remote
Categories
AI Agent Engineer
About the job
As Engineering Manager for Agent Oversight, you will lead the team responsible for building Scale's platform for monitoring, evaluating, and improving agentic applications for enterprise and government clients. This role involves owning the platform's roadmap, encompassing deployment, monitoring, evaluation, ML-driven improvement, and continuous learning to ensure agents enhance over time. You will manage the engineering team, drive technical delivery, and collaborate with customers and internal partners to bring these critical capabilities to production, aiming to set the industry standard for agentic application oversight. The platform currently supports agent deployments for multiple clients, and this US-based role will play a key part in defining and scaling the team.
Responsibilities
- Lead a multi-disciplinary team of software and ML engineers to drive technical delivery across the Scale Generative AI Platform (SGP).
- Own the platform's roadmap for deployment, monitoring, evaluation, and ML-driven improvement of agentic applications.
- Collaborate with customers, forward-deployed teams, product, and internal engineering teams to translate requirements into platform capabilities.
- Build and ship features end-to-end, from system design through debugging and testing.
- Drive high-velocity experimentation to validate and improve platform capabilities based on real customer usage.
- Establish the technical direction, culture, and processes for a fast-growing team.
- Mentor and develop engineers and ML engineers/scientists, influencing team scaling.
Requirements
- 7+ years of engineering experience, with at least 2 years managing engineers or ML engineers responsible for production ML/LLM-powered systems.
- Hands-on familiarity with agent architectures (tool use, planning, multi-agent orchestration) and the technical depth to make informed tradeoffs.
- Ability to review ML experiment design or evaluation methodology to ask sharp questions and gain credibility with ML engineers and scientists.
- Track record of owning the full lifecycle of platform-level infrastructure, from design to scaling.
- Experience collaborating with product managers, forward-deployed engineering teams, and customers to translate requirements into shipped capabilities.
- Track record of building and growing high-performing engineering teams, including hiring, retention, or measurable improvements in team output.
Benefits
- Base salary
- Equity
- Benefits