Evals Jobs

This internship program offers a full-time, in-person opportunity in Berlin for 12 to 24 weeks. Interns will focus on advancing search quality through innovative models, data, and tools. The role involves training and optimizing large-scale deep learning models, with a particular emphasis on retrieval and ranking models. Additionally, interns will conduct research in representation learning, covering areas like contrastive learning, multilingual, evaluation, and multimodal modeling for search and retrieval. A key aspect of the role includes building and optimizing RAG pipelines for enhanced grounding and answer generation.

RevenueCat is seeking a Senior Software Engineer to focus on building advanced AI agents that empower app developers to manage subscriptions and grow revenue. You will work on agentic features that handle orchestration, tool execution, context management, evaluation, and human-in-the-loop controls. This role involves shipping impactful features to thousands of developers and shaping the foundational infrastructure for these AI agents. You will be part of a remote-first team dedicated to customer obsession, shipping, ownership, and balance, working on systems that manage billions of dollars and impact hundreds of millions of users.

$230k - $230k

Anthropic

Anthropic's Safeguards organization is responsible for developing the policies, evaluations, and enforcement systems that prevent AI models from causing catastrophic harm. We are seeking a manager to lead our biological safety research engineering team. This team focuses on creating the evaluations, datasets, and classifiers that govern how our models handle biological knowledge. You will guide a team of research scientists and engineers in designing and executing capability evaluations on frontier models, curating safety classifier training data, and training/iterating on these classifiers. Your role involves setting technical direction, making investment decisions, and owning team outcomes, while maintaining technical depth to review designs and analyze failure modes. The team's core challenge is balancing robust safeguards against sophisticated actors with enabling legitimate researchers in the life sciences.

$405k - $485k

Mercor is seeking a Member of Technical Staff to build the backend systems and agent infrastructure for their talent network. A significant portion of Mercor's operations rely on agents for sourcing, screening, matching, and supporting experts with minimal human intervention. This role involves designing and deploying the services that power these agents, including orchestration, tool interfaces, evaluation, and data models for auditable and reversible decisions. This is a hands-on engineering position requiring coding, end-to-end system ownership, and collaboration with product, operations, and AI researchers to define agent functionalities and human involvement. The company is a profitable Series C organization valued at $10 billion, focused on organizing human intelligence to power the AI economy. They are a leading AI data company building the interface between human expertise and advanced AI models, with millions of domain experts earning significant daily income to train frontier AI models. Mercor's APEX benchmark family measures AI's real-world impact on professional work, and Mercor Enterprise extends this infrastructure to Fortune 500 companies.

$160k - $500k