Research Scientist, Agent Robustness
$216k - $270k • Remote • San Francisco, CA; New York, NY
Posted 8d ago
Job Location
San Francisco, CA; New York, NY
Tech Stack
Remote Work Policy
Fully remote
Categories
AI Agent Engineer
About the job
Scale Labs is launching a new team focused on policy research to bridge the gap between AI research and global policymakers, enabling informed, scientific decisions about AI risks and capabilities. This team tackles fundamental challenges in agent robustness, AI control protocols, and AI risk evaluations to help governments, industry, and the public understand and mitigate AI risk while maximizing AI adoption. The role involves collaborating broadly across industry, the public sector, and academia, and regularly publishing findings to shape the future of AI safety and alignment.
Responsibilities
- Research the science of AI agent capabilities, focusing on safety, risk factors, and benchmarking methodologies.
- Design and build systems to test AI agents' tendency to take harmful actions under pressure or deception.
- Design and build exploits and mitigations for failure modes in AI agents with advanced affordances (coding, web browsing).
- Characterize and design mitigations for potential failure modes in systems involving multiple interacting AI agents.
Requirements
- Commitment to promoting safe, secure, and trustworthy AI deployments.
- Practical experience conducting collaborative technical research.
- Comfortable building and leveraging agent scaffolding and designing evaluation harnesses.
- Ability to quickly prototype new ideas from research literature.
- Experience with post-training and RL techniques such as RLHF, DPO, GRPO.
- Track record of published research in machine learning, particularly generative AI.
- At least three years of experience addressing sophisticated ML problems.
- Strong written and verbal communication skills for cross-functional teams.
- Hands-on experience with agent evaluation frameworks (e.g., SWE-bench, WebArena).
- Experience with red-teaming, prompt injection, or adversarial testing of AI systems.
Benefits
- Base salary
- Equity
- Comprehensive health, dental and vision coverage
- Retirement benefits
- Learning and development stipend
- Generous PTO
- Commuter stipend (potential)