LLM-as-judge Jobs

FurtherAI is building AI agents to revolutionize the insurance industry, a trillion-dollar market currently reliant on manual processes. We combine frontier agentic AI with deep domain expertise to transform critical industry operations. Having secured over $30 million in funding from top investors and experienced significant growth, we are a deeply technical company where engineers have a substantial impact on product direction. We are seeking an exceptional AI Engineer to lead our efforts in evaluating and continuously enhancing the quality of AI agents across our platform.

$300k - $350k

Code Intelligence is at the intersection of agentic engineering and product, running agents against real customer source code in production. This role offers the opportunity to lead an AI-native team in a business-critical growth area, providing high visibility and leverage within the fastest-growing segment of the AI agent space. Datadog operates as a hybrid workplace, valuing office culture for collaboration and creativity while ensuring employees can achieve work-life harmony.

$192k - $240k

Figma is seeking a Director, Research - AI Evals to lead the measurement of quality for Figma's AI-powered experiences. This role is crucial as Figma integrates more AI capabilities, requiring a rigorous approach to defining and assessing "good" AI features. You will establish frameworks, quality standards, and trusted signals that guide product teams in their AI feature development and release decisions. The ideal candidate possesses deep, hands-on experience in evaluating AI/LLM products, combining human and automated methods, with strong product intuition and communication skills to ensure evaluation is impactful. You will collaborate with Product, Design, Engineering, and Data Science teams, influencing AI shipping decisions that affect millions of users.

$258k - $348k