Research, Safety
$350k - $475k • San Francisco • FullTime
Posted 1mo ago
Job Location
San Francisco
Tech Stack
Remote Work Policy
On-site
Employment Type
FullTime
Categories
AI Research Engineer
About the job
Thinking Machines is building AI that extends human will and judgment, focusing on training frontier models, enabling user customization, and crafting interfaces for human-AI communication. As a safety researcher, you will ensure our models are safe and trustworthy by investigating how models handle harmful requests, what they learn from data, and how training influences their behavior. This role involves both research and hands-on technical work, exploring the science behind model behaviors and designing experiments to guide training and evaluation processes.
Responsibilities
- Build data filtering pipelines and quality classifiers to shape model learning from pre-training corpora and study their impact on downstream safety behavior.
- Apply post-training techniques like RLHF/RLAIF and policy-based reasoning to guide model handling of harmful, sensitive, and dual-use requests.
- Design, build, and maintain safety evaluations, focusing on measuring model behavior on long-horizon and agentic tasks.
- Generate and curate synthetic data for training and evaluating models on refusal boundaries and safety-relevant behaviors.
- Red-team models and products to identify failure modes, jailbreaks, and emergent risks before deployment, and design mitigations.
Requirements
- Bachelor’s degree or equivalent experience in Computer Science, Machine Learning, Physics, Mathematics, or a related discipline with strong theoretical and empirical grounding.
- Background in AI safety research with hands-on experience in at least one area: RLHF/RLAIF, alignment and preference modeling, deliberative alignment, safety evaluations, or red-teaming.
- Proficiency in Python and familiarity with deep learning frameworks (e.g., PyTorch, TensorFlow, JAX).
- Comfort debugging distributed training and writing scalable code.
- Clarity in communication and ability to explain complex technical concepts in writing.
- Experience building evaluations for long-horizon, multi-step, or agentic tasks (preferred).
- Experience generating synthetic data at scale for training or evaluation (preferred).
- Experience with modern red-teaming/jailbreaking techniques (preferred).
- Research contributions in AI safety (publications, open-source evaluations, public red-teaming work) (preferred).
- Familiarity with AI safety literature and current open problems (e.g., scalable oversight, reward hacking, jailbreak robustness) (preferred).
- PhD in Computer Science, Machine Learning, Physics, Mathematics, or a related discipline with strong theoretical and empirical grounding; or equivalent industry research experience (preferred).
Benefits
- Generous health, dental, and vision benefits
- Unlimited PTO
- Paid parental leave
- Relocation support