Research Scientist, Agent Robustness
$216k - $270k • San Francisco, CA; New York, NY
Posted 2mo ago
Job Location
San Francisco, CA; New York, NY
Tech Stack
Remote Work Policy
On-site
Categories
AI Agent Engineer
About the job
Scale Labs is seeking talented researchers to join a new team focused on policy research, bridging the gap between AI research and global policymakers to make informed, scientific decisions about AI risks and capabilities. This role will tackle fundamental challenges in building AI agents that are safe and aligned with humans, researching agent capabilities, designing evaluation harnesses, building exploits and mitigations for failure modes, and characterizing risks of multi-agent systems. The team collaborates broadly across industry, the public sector, and academia, regularly publishing findings.
Responsibilities
- Research the science of AI agent capabilities, focusing on safety, risk factors, and benchmarking methodologies.
- Design and build harnesses to test AI agents' tendency to take harmful actions under pressure or deception.
- Design and build exploits and mitigations for failure modes arising from AI agent affordances like coding and web browsing.
- Characterize and design mitigations for potential failure modes or broader risks in systems with multiple interacting AI agents.
Requirements
- Commitment to promoting safe, secure, and trustworthy AI deployments.
- Practical experience conducting collaborative technical research, including building agent scaffolding, designing evaluation harnesses, and prototyping from research literature.
- Experience with post-training and RL techniques such as RLHF, DPO, GRPO, and similar approaches.
- A track record of published research in machine learning, particularly in generative AI.
- At least three years of experience addressing sophisticated ML problems in research or product development.
- Strong written and verbal communication skills for cross-functional team collaboration.
Benefits
- Base salary
- Equity
- Comprehensive health, dental and vision coverage
- Retirement benefits
- Learning and development stipend
- Generous PTO
- Commuter stipend