Research, RL Scaling
$350k - $475k • San Francisco • FullTime
Posted 1mo ago
Job Location
San Francisco
Tech Stack
Remote Work Policy
On-site
Employment Type
FullTime
Categories
AI Research Engineer
About the job
Thinking Machines is seeking a researcher to scale reinforcement learning for frontier models. This role focuses on the intersection of training recipes and infrastructure, particularly for high-training-compute, long-horizon RL. A key area of interest is asynchronous RL, requiring a deep understanding of its algorithms, design choices, and trade-offs on both ML and systems sides. The position also emphasizes the importance of inference systems, low-precision numerics, and quantization, as these significantly impact RL scaling and cost. This is a research role with full-stack ownership, from algorithms to training run health.
Responsibilities
- Co-design RL recipes and their supporting systems, validating choices at frontier scale.
- Advance asynchronous RL algorithms.
- Improve efficiency of rollout generation and its integration with training.
- Manage frontier-scale RL end-to-end, including model bring-up and stability.
- Jointly optimize compute and training efficiency for RL, focusing on accelerators, memory, communication, and low-precision numerics.
- Conduct rigorous empirical science, including ablations and scaling studies with reliable instrumentation.
Requirements
- Proficiency in Python and at least one deep learning framework (PyTorch, TensorFlow, JAX).
- Experience debugging distributed training and writing scalable code.
- Bachelor’s degree or equivalent experience in a quantitative field (CS, ML, Physics, Math) with strong theoretical and empirical grounding.
- Clear communication skills, with the ability to explain complex technical concepts in writing.
- Strong research judgment, including clean ablations, honest baselines, and clear technical writing.
- PhD or equivalent industry research experience is preferred.
- Strong grounding in RL for large language models, including modern policy optimization methods.
- Deep understanding of asynchronous RL algorithms, design choices, and trade-offs.
- Experience training large models across many accelerators and comfort with distributed systems.
- Good working knowledge of inference systems and quantitative reasoning about rollout generation.
- Experience building or operating decoupled generation/training RL systems at scale.
- Familiarity with low-precision training and inference (numerics, quantization).
- Hands-on work with LLM serving stacks (e.g., SGLang, vLLM).
- Experience with scaling studies for large models.
- Contributions to open-source training or inference frameworks are a plus.
Benefits
- Health, dental, and vision benefits
- Unlimited PTO
- Paid parental leave
- Relocation support