Evals Jobs

Harvey is transforming legal and professional services by combining frontier agentic AI, an enterprise-grade platform, and deep domain expertise. This role is an opportunity to join a fast-scaling company at an inflection point, defining a new category in real-time. The team operates with intensity, ownership, and a commitment to excellence, guided by values of Decisiveness, Simplicity, and Job's Not Finished. We are looking for driven individuals to do the best work of their careers alongside like-minded colleagues.

$230k - $340k

Harvey is transforming legal and professional services by combining frontier agentic AI, an enterprise-grade platform, and deep domain expertise. This role is an opportunity to join a fast-scaling company at an inflection point, defining a new category in real-time. The team operates with intensity, ownership, and a commitment to excellence, guided by values of Decisiveness, Simplicity, and Job's Not Finished. We are looking for driven individuals to do the best work of their careers alongside like-minded colleagues.

$230k - $340k

Distyl AI is seeking an AI Engineer to build and operate production AI systems that deliver business value within customer environments. This role is for engineers who excel in ambiguous problem spaces, take ownership of outcomes, and are eager to work directly on AI systems that must perform reliably under enterprise constraints. AI Engineers are hands-on builders who design, implement, deploy, and iterate on end-to-end AI systems in close partnership with customers, subject matter experts, and other Distyl engineers. They translate operational needs into concrete system behavior, build the necessary software and AI workflows, and continuously improve systems through evaluation, feedback, integration, and production iteration. This is a practical role focused on making AI systems work effectively in real-world scenarios with users, data, and accountability for production outcomes.

$180k - $250k

Dscout is seeking a Senior Forward Deployed Engineer to join their AI Training & Evals team. This new unit focuses on generating human data and evaluations to ensure AI models perform effectively for real users in real-world contexts. You will partner with Strategic Project Leads to translate customer needs into technical plans, build necessary components, and ensure the quality and delivery of data for AI training and evaluation. This role offers the opportunity to work on cutting-edge AI problems, build on an existing mature platform, and directly influence product development by identifying customer needs and contributing to the core platform.

$250k - $400k

Dscout is seeking an AI Research Engineer to join their new AI Training & Evals unit. This team focuses on building human data and evaluations to determine if AI performs effectively for real people in real-world contexts. While AI models may excel at benchmarks, they often surprise users in practice. This role bridges that gap by leveraging Dscout's decade of experience in capturing human behavior to create training data, evaluations, and benchmarks that AI teams can trust. The position offers a unique opportunity to work on unsolved problems in AI measurement, study real participants in their own context, and contribute to publishing novel methods and benchmarks.

$250k - $400k

Figma is seeking an Engineering Manager for Design Systems Adherence to lead a team focused on building workflows, tools, and feedback loops that ensure customers' design and code correctly adhere to their design systems when using Figma's Agents. This role involves enhancing context engineering for AI, improving agent architecture, and developing datasets and evaluators to measure adherence. The manager will set strategy, elevate adherence measurement standards, and translate evaluation results into actionable improvements for design quality. This is a high-impact opportunity to influence Figma's Agent capabilities and overall design quality, addressing the critical need for design and code parity and minimizing drift from defined design systems.

$258k - $376k

The Radar ML team builds the machine learning systems that power Stripe’s Radar fraud and abuse product suite. Our models make real-time decisions across a significant share of internet commerce, helping businesses stop fraud, increase authorization, and grow safely. By turning economy-scale datasets into compelling ML products, the team directly improves customer outcomes and drives meaningful revenue for Stripe. The team is growing and expanding into multiple new areas, including building abuse protections across the full customer life-cycle, adaptive and customizable fraud systems for enterprises, and risk intelligence for over 50 global payment methods. You will own major ML systems and initiatives from problem definition to production, finding where ML can make the biggest difference, designing the approach, building and shipping models and the systems around them, and continuously improving them as data, users, and attackers change. Your work may involve protecting businesses from abuse across the customer lifecycle, building adaptive and customizable fraud systems for Stripe's largest users, or extending risk intelligence to over 50 global payment methods. Across all these, you'll use modern approaches such as transformer models over payment data, graph intelligence across consumers, and LLMs to detect emerging attacks. You will have opportunities to ship 0→1 products and improve mission-critical ML models operating at Stripe scale, often driving several efforts in parallel and setting the technical bar for the team.

The Radar ML team builds the machine learning systems that power Stripe’s Radar fraud and abuse product suite. Our models make real-time decisions across a significant share of internet commerce, helping businesses stop fraud, increase authorization, and grow safely. By turning economy-scale datasets into compelling ML products, the team directly improves customer outcomes and drives meaningful revenue for Stripe. The team is growing and expanding into multiple new areas, building abuse protections across the full customer life-cycle, adaptive and customizable fraud systems for enterprises, and risk intelligence for over 50 global payment methods. You will own ML problems from conception to production, analyzing new fraud and abuse patterns, experimenting with model architectures, designing evaluations, deploying models, and monitoring performance. Your work may include developing transformer architectures for payment data, building graph-powered intelligence, leveraging LLMs to detect emerging attacks, or rapidly prototyping new protections.

Moveworks is seeking a Senior Staff ML Engineer to join our AI Safeguards team. In this role, you will be instrumental in designing, building, and shipping critical safeguards within our agent harness. Your work will focus on protecting against prompt injection, single and multi-turn jailbreaks, toxicity, sensitive topics, and sensitive data exposure. You will develop robust evaluation frameworks to measure the efficacy of these safeguards, addressing both false positives and false negatives using diverse datasets. Collaboration with our Security team will be key, translating their findings into actionable eval cases and fixes, and co-developing mitigations. You will also select, integrate, and fine-tune detection models within a shared stack serving both commercial and government environments, while carefully balancing quality, latency, and cost for controls operating in the request path. Additionally, you will build configurable safeguard policies to empower customers in managing sensitive topics and monitor safeguard behavior in production, addressing any regressions.

Moveworks is seeking a Senior Staff Security Engineer to focus on AI Safeguards. In this role, you will be responsible for maintaining an up-to-date map of ML usage and safeguards across our products, and leading the security design for new ML features. You will actively test the boundaries of our safeguards against various attacks like jailbreaks, prompt injection, and toxicity, translating these findings into evaluation cases that are applied to every model, prompt, and safeguard change. Additionally, you will collaborate with ML engineers to design and build robust mitigations that remain effective even when detection fails, and define and report on key security metrics to demonstrate the efficacy of our safeguards. You will also stay abreast of the latest AI attack research, incorporating new techniques into our testing and mitigation strategies.

Turing's mission is to accelerate superintelligence to drive real economic progress. The company works with frontier AI labs to generate high-quality datasets, reinforcement learning environments, and frontier research benchmarks. Turing also partners with Fortune 500 enterprises to build and deploy end-to-end agentic AI systems within mission-critical workflows. This role involves leading the Turing Enterprise AI Platform technical team, which is being built out for a late 2025 SDK release and a Q1 2027 tenant onboarding. The goal is to grow the team to eight to ten members and collaborate with the Chief AI Architect on client-facing architecture and evaluation.

As a multimodal engineer on the Imagine Model Team, you will develop cutting-edge AI experiences beyond text, with a strong focus on enabling high-fidelity understanding and generation across image and video modalities, while also incorporating audio where it enhances visual content. Responsibilities span data curation, modeling, training, inference serving, and product integration, covering both pretraining and post-training phases. You will collaborate closely with product teams to push model frontiers and deliver exceptional end-to-end user experiences.

$180k - $440k

Raindrop is seeking a founding Research Engineer to investigate the behavior of AI agents in production environments. This role is crucial for an early-stage team, involving setting the research direction, conducting experiments, and disseminating findings through publications. The focus will be on understanding agent failures, advancing simulation techniques, developing novel detection methods for undefined issues, and exploring high-impact open questions in AI agent behavior. The ideal candidate will possess strong communication skills, a passion for sharing research, deep expertise in areas like interpretability or AI safety, and the ability to prototype and ship experiments.

Accenture

We are seeking a forward-thinking professional with an AI-first mindset to design, develop, and deploy enterprise-grade solutions using Generative and Agentic AI frameworks that drive innovation, efficiency, and business transformation. This role involves leading AI-driven solution design and delivery by applying GenAI and Agentic AI to address complex business challenges, automate processes, and integrate intelligent insights into enterprise workflows for measurable impact. The position also requires designing and maintaining Data Vault 2.0 models for a specific domain or subject area within a larger enterprise data platform, engaging with key business representatives, data owners, end users, data architects, and ETL/ELT developers to model data and ensure alignment with best practices and industry standards.

Mercor is seeking a Research Engineer Intern to work at the intersection of engineering and applied AI research. This role will focus on owning benchmarking pipelines, evaluation systems, and failure analysis workflows that directly inform how we train and improve frontier language models. Your work will be crucial in defining how we measure tool use, agentic behavior, and real-world reasoning. You will design and run evaluations, build rubrics and scorers, and translate failure analysis into actionable improvements for post-training, RLVR, and data pipelines.

Discovery is seeking an AI & Agentic Platform Engineer to join their expeditionary force. This role involves taking unproven concepts and building the analysis or prototype to test them, working on challenges across autonomy, sensor analysis, and battle space management. The team focuses on moving AI models from research into production, specifically onto classified platforms and edge hardware where reliability is paramount. Rigorous and repeatable evaluation of model performance is critical as the model portfolio and classified work expand.

$260k - $345k

We are seeking an experienced software engineer to join our Agentic AI Harness & Quality team, focusing on expanding customer-facing agentic AI capabilities. You will leverage modern agentic AI tools, including frontier LLMs and infrastructure for tuning, evaluating, and serving AI agents in production. This is an eval-driven, data-centric role where you'll contribute to building error-free, inclusive, and privacy-preserving datasets. Your work will focus on enhancing AI performance in production, encompassing accuracy, quality, latency, reliability, and the overall user-facing system. We aim to build the world's highest-performing enterprise assistant platform, and your contributions will be crucial to this objective.

Turing is seeking a Senior Software Engineer to join their AI Evaluation Infrastructure team. This role focuses on building the core infrastructure that frontier AI labs use to evaluate coding agents and language models. You will be responsible for designing and implementing features across the entire stack, from data models and background jobs to user-facing interfaces. The ideal candidate is an experienced, independent engineer who can translate loosely defined requests into robust, tested, and reliable solutions. This position offers the opportunity to work on cutting-edge AI evaluation tools and contribute to the advancement of AI capabilities.

$170k - $200k

OpenAI

OpenAI's Forward Deployed Engineering team partners with customers to turn research breakthroughs into production systems, operating at the intersection of customer delivery and core platform development. As a Forward Deployed Engineer for AI Infra, you will lead complex deployments of frontier models in production, embedding with customers where model performance and urgent delivery are critical. You will work directly with customer engineers, researchers, and product engineers on optimization problems across semiconductors and AI infrastructure, writing and debugging software, running experiments, and improving tools. Your role involves defining problems, structuring delivery, and shipping fast, with a focus on inference performance and chip-design optimization, ultimately contributing improvements to OpenAI's platform.

$185k - $300k

NVIDIA is seeking a Senior Staff AI Engineer to join its Finance AI team. In this role, you will be responsible for designing, building, evaluating, and operating AI-powered applications, agents, and automation critical to Finance operations. You will lead a team of coding agents, integrate new tools and technologies, and apply deep business knowledge to solve complex problems with AI. This is primarily an applied engineering position focused on leveraging foundation models, with opportunities to demonstrate the advantages of customized models over standard prompting and retrieval techniques.

$200k - $322k