1 open role mentioning Measurement
Datadog
The Agentic Interfaces team at Datadog is seeking a Staff Applied Scientist to lead the evaluation strategy for AI agent integrations. This role focuses on defining and building the measurement systems that ensure AI agents interacting with Datadog's observability, security, and software delivery data are continuously improving. You will be responsible for establishing metrics that cover answer quality, tool selection accuracy, retrieval relevance, latency, and cost, and will partner with AI engineers to implement these systems. This is an opportunity to tackle open research questions in agent-data interaction, such as evaluating non-deterministic trajectories, scoring tool selection in large catalogs, and building scalable measurement systems that benefit both first-party and third-party agents.
$276k - $345k