Research Scientist, Applied White-Box Methods
$150k - $250k • Berkeley Office • Full-time
Posted 1mo ago
Remote Work Policy
On-site
Employment Type
Full-time
Categories
AI Research Engineer
About the job
FAR.AI is a non-profit AI research institute dedicated to ensuring advanced AI is safe and beneficial. The Applied White-Box Methods team focuses on developing, evaluating, and demonstrating techniques that utilize model internals to enhance AI system safety. This involves exploring applications like white-box control, evaluation awareness, and shaping training dynamics for improved alignment. The team bridges the gap between exploratory research and practical deployment by rigorously testing white-box methods on real-world tasks, such as long-context agentic coding, and using feedback to refine these methods for frontier-scale deployment. The scope encompasses any method using model internals for understanding, predicting, or intervening in model behavior, including activation explainers, steering, patching, influence functions, and singular learning theory. These methods are evaluated against strong black-box baselines to ensure their practical value.
Responsibilities
- Take ownership of and accelerate the team's research agenda.
- Publish research findings broadly.
- Engage with the AI alignment community.
- Propose new research directions within the team's agenda.
- Leverage FAR.AI's infrastructure for events and government relations.
- Collaborate with national AI safety institutes, frontier model developers, and top academics.
- Develop, evaluate, and demonstrate methods that leverage model internals for AI safety.
- Stress-test white-box methods on real-world tasks.
- Enable the deployment of better white-box methods at frontier scale.
- Study long-context agentic coding, reward hacking, sandbagging, and research tampering.
- Ensure new methods are simple and efficient enough for frontier-scale deployment.
Requirements
- Experience in fundamental interpretability with a desire to evaluate and refine methods in realistic settings (agentic coding, long contexts, realistic threat models).
- Experience in evaluations, AI control, red-teaming, or reinforcement learning with a focus on model internals.
- Hands-on experience applying at least one white-box method (e.g., activation explainers, SAEs, steering, attribution, influence functions, probes) to a real model.
- Demonstrated interest in the practical application of white-box methods.
- Ability to explain how previous research experience (e.g., applied ML) can be leveraged for AI safety work.
- Engagement with the field of technical AI safety research.
- Experience with realistic, large-scale models and evaluations.
- Familiarity with Goodhart-resistant metrics for AI research automation.