Evals Jobs

Salesforce is seeking a Software Engineering, AI Systems professional to join Salesforce AI Labs. This role involves building a product that learns from human demonstrations to automate complex workflows across enterprise applications. You will tackle challenging engineering problems related to agent dependability, reasoning, and error recovery. The position offers the opportunity to work directly with customers, own substantial systems from design to deployment, and contribute to taking a new offering from early adoption to general availability, supported by Salesforce's scale.

$149k - $224k

OpenAI

The Agent Post-Training team at OpenAI is responsible for developing the frontier agents that are shipped to users. This involves training models for products like Codex and ChatGPT, focusing on creating persistent, proactive intelligence that can operate computers, collaborate with humans and other agents, and expand user capabilities. The team defines the next generation of agent abilities, builds the training signals, and runs experiments to realize these advancements. The role specifically focuses on teaching models to operate computers, including navigating browsers and desktops, using tools, reasoning through complex workflows, and completing long-horizon tasks reliably. This position is at the intersection of frontier model training, product behavior, evaluation, and systems engineering, directly influencing the computer-use capabilities of OpenAI's future agents.

$380k - $500k

Prime Intellect is building the open superintelligence stack, a platform designed to unify compute, environments, evaluations, and high-performance training for AI labs. Their platform, Lab, supports full-stack systems for post-training at frontier scale, including SFT, RL, tool use, agent workflows, and continuous model improvement. They are focused on creating open-source models for long-horizon tasks like autonomous research and the platform their own research team uses. The company aims to provide AI companies and research teams with the ability to transform their workflows, tools, and data into owned superintelligence.

$150k - $300k

Prime Intellect is building the open superintelligence stack, providing infrastructure for AI labs. Their platform, Lab, unifies compute, environments, evaluations, and training for post-training at scale, including SFT, RL, tool use, and agent workflows. They are focused on open-source models trained for long-horizon tasks and the platform used to build them. The company has raised significant funding from prominent investors and is seeking individuals to work at the intersection of frontier research, infrastructure, and go-to-market strategy for a nascent category.

$150k - $300k

Prime Intellect is building an open superintelligence stack, providing infrastructure for AI labs. Their platform, Lab, unifies compute, environments, evaluations, and training for post-training at scale, including SFT, RL, tool use, agent workflows, and continuous model improvement. They are focused on open-source models for long-horizon tasks and the platform their research team uses. The company has raised significant funding from prominent investors and is looking for individuals passionate about building at the intersection of frontier research, infrastructure, and go-to-market strategy for a nascent category.

$150k - $300k

Deepgram is a leading platform for Voice AI, providing real-time APIs for speech-to-text and text-to-speech, enabling developers to build production-grade voice agents at scale. We are seeking researchers who are passionate about solving customer challenges by improving our AI models. This role involves identifying model limitations, designing evaluations, curating data, fine-tuning models, and shipping solutions to production within weeks. The ideal candidate will have an AI-first mindset, be comfortable experimenting with new AI tools and models, and thrive in a fast-paced, evolving environment.

$216k - $338k

Alpaca is seeking a Senior Data Scientist, AI Evaluation to lead the measurement of our AI models and agents. In this senior individual contributor role, you will transform ambiguous quality questions into reliable ground truth, scoring methods, and evaluation loops that drive company-wide improvements. You will leverage an existing data foundation to enhance quality and accelerate the safe rollout of AI systems, owning the quality bar independently. This role is ideal for someone passionate about the correctness of AI outputs as much as their generation, who can partner with Product, Engineering, Analytics Engineering, and business stakeholders to define success, build robust evaluations, and ensure feedback loops drive iteration. If you have a strong quantitative background, a proven track record of rigorous, measurable work in evaluation, experimentation, or model validation, and desire ownership of a new evaluation practice at a rapidly growing financial services infrastructure company, we encourage you to apply.

NVIDIA is seeking a founding Director to establish and lead its Forward Deployed Engineering (FDE) organization for open models, specifically focusing on NVIDIA Nemotron and NVIDIA Cosmos across North America. The core mission is to transform open-source models into production-ready AI solutions tailored to specific domains, leveraging NVIDIA's partner ecosystem. This role involves setting the strategic direction for North America, building and scaling the FDE team, and fostering strong relationships with partners, customers, and NVIDIA leadership. The FDE team will work collaboratively with top partner engineering and research teams, co-leading joint customer engagements and developing best practices to ensure partner success and scalability. The ultimate goal is to empower organizations to build, own, and operate their AI on open-source models.

$320k - $489k

Decagon is a leading conversational AI platform that empowers brands to deliver exceptional customer experiences through AI agents. Our technology enables enterprises to deploy personalized and satisfying interactions across various channels. We are building a future where customer experiences are redefined, moving beyond traditional support to foster faster resolutions, richer conversations, and deeper relationships. The Product Engineering team at Decagon is responsible for building the platform that allows customers to create, operate, and enhance their AI agents, transforming complex engineering tasks into user-friendly product features. This team is crucial for scaling Decagon's impact, with the frameworks they build supporting every agent in production, ensuring reliability and robustness across diverse workflows and constraints. They also build agents directly, using production failures as opportunities to create guardrails, evaluations, and tooling that improve the platform universally.

$200k - $400k

Decagon is seeking a Senior Software Engineer to join the Product Engineering team. This team is responsible for building the core platform that enables customers to create, operate, and improve their AI agents. You will design and build systems that allow customers to configure, train, test, and monitor sophisticated AI agents with confidence, translating complex technical capabilities into simple, usable interfaces. You will own features end-to-end, from technical architecture through deployment and iteration, working closely with product and design to understand customer use cases and ensure the right solutions are built. This role involves building self-serve functionality, designing configuration abstractions, creating AI-powered tooling for agent workflows, and developing monitoring and analytics for agent performance.

$200k - $400k

Decagon is a leading conversational AI platform that empowers brands to deliver exceptional customer experiences through AI agents. Our technology enables enterprises to deploy personalized and satisfying interactions across various channels. We are building a future where customer experiences are redefined, moving beyond traditional support to foster faster resolutions, richer conversations, and deeper relationships. The Product Engineering team at Decagon is responsible for building the platform that allows customers to create, operate, and enhance their AI agents, transforming complex engineering tasks into user-friendly product features. This team is crucial for scaling Decagon's impact, with the frameworks they build supporting every agent in production, ensuring reliability and robustness across diverse workflows and constraints. They also build agents directly, using production failures as opportunities to create guardrails, evaluations, and tooling that improve the platform universally.

$200k - $400k

Decagon is seeking a Senior Software Engineer to join the Product Engineering team. This team is responsible for building the core platform that enables customers to create, operate, and improve their AI agents. You will design and build systems that allow customers to configure, train, test, and monitor sophisticated AI agents with confidence, translating complex technical capabilities into simple, usable interfaces. You will own features end-to-end, from technical architecture through deployment and iteration, working closely with product and design to understand customer use cases and ensure the right solutions are built. This role involves building self-serve functionality, designing configuration abstractions, creating AI-powered tooling for agent workflows, and developing monitoring and analytics for agent performance.

$200k - $400k

AssemblyAI is seeking a Software Engineer to own the agent experience (AX) for developers building with their Voice AI models. This role focuses on ensuring AI coding agents can seamlessly discover, integrate, and debug AssemblyAI products. You will be responsible for developing agent skills, prompts, and integrations that enable agents to create production-grade AssemblyAI integrations on the first try. This is a unique opportunity to shape a new and rapidly growing area within a successful AI company, working closely with senior engineers and gaining significant ownership from day one.

$160k - $210k

This role focuses on establishing and implementing a comprehensive evaluation strategy for autonomous agents at NVIDIA. The goal is to ensure that changes to agents, their underlying models, harnesses, tools, and skills lead to improvements rather than regressions. You will design systems that allow agents to leverage evaluation results for self-improvement, building trust in the process of autonomous evolution. This involves creating robust evaluation frameworks, mining production data for insights, and developing realistic simulation environments for safe, end-to-end agent task execution.

$224k - $431k

Our Internal Acceleration team is seeking a Research Engineer to build agents that accelerate the production of frontier tokens. You will collaborate directly with founders and embed with the production team to develop new agentic harnesses and operating systems, transforming hardware development in an AI-native world. This role involves working on harnesses, tool use, context, memory, execution environments, and evaluation frameworks to enable agents to perform long-running, useful engineering tasks. We are looking for individuals who thrive in ambiguous environments, possess strong product sense, and are eager to own problems end-to-end, viewing this as an opportunity to shape future ventures. The work is time-sensitive and business-critical, demanding dedication beyond a standard 9-5 schedule.

$150k - $400k

Our Internal Acceleration team is building agents to speed up the production of frontier tokens. You will collaborate directly with founders and embed with our special projects team to develop new agentic harnesses and operating systems, fundamentally changing how hardware is built in an AI-native world. Your success will be measured by the acceleration of Etched's progress, not just lines of code. This role is ideal for engineers who excel in ambiguous environments, possess strong product intuition, and are eager to own problems from inception to completion. It's a dynamic, time-sensitive, and business-critical position that requires dedication beyond a standard 9-5.

$150k - $400k

Our Internal Acceleration team is building agents to accelerate the production of frontier tokens. You will work directly with founders and embed with our kernels team to develop new agentic harnesses and operating systems, transforming hardware development in an AI-native world. This role focuses on harnesses, tool use, context, memory, execution environments, and evaluations that enable agents to perform long-running, useful engineering work. We seek engineers who thrive in ambiguity, possess strong product sense, and are eager to own problems end-to-end, understanding that this is a time-sensitive and business-critical role that may not adhere to a 9-5 schedule.

$150k - $400k

SpaceXAI is building Super Intelligence to help humanity understand the universe, from supercomputers to large language models and user-facing applications. As a Software Engineer on the Model Lifecycle and Signals team, you will be responsible for integrating frontier models into user-facing products at a massive scale. This role involves deciding which models to use, monitoring their performance, and providing feedback to shape future model development. You will work at the intersection of infrastructure, data, and product, bringing models from preview to production, analyzing real-world data, and collaborating with research, inference, product, and data science teams to improve both products and models. Your contributions will have a compounding impact across all products and shipped models.

Salesforce is seeking a technical leader for its AI Core team to build durable, high-quality systems that address complex AI challenges. This role involves developing retrieval systems and agent infrastructure, enabling AI agents to effectively find, trust, and utilize knowledge. You will also create developer tooling to help engineers across the company rapidly and safely deploy production agents. The position offers end-to-end ownership, from scoping and implementation to deployment and iterative improvement, within a high-ownership, high-velocity environment focused on AI and product development.

$173k - $260k

Salesforce is seeking a Software Engineering SMTS for the Brain Team, focusing on the intelligence layer behind their AI CRM product, Piper. This role involves building, maintaining, and upgrading AI agents that qualify leads and book meetings. You will work on system prompts, model configuration, context layers, evaluations, and the underlying harness that powers Piper. The position offers a high-ownership, high-velocity environment where engineers ship customer-facing AI products at scale, with end-to-end ownership from idea to production. You will collaborate with product and AI platform teams to develop features, enhance customer experiences, and build evaluation systems to ensure agent performance and business outcomes.

$149k - $224k