Software Engineer - RL Environments
$260k - $290k • San Francisco • FullTime
Posted 6mo ago
About the job
AfterQuery is an applied research lab focused on curating data solutions for foundation model development, serving frontier AI labs with the mission of delivering the best data to power the best models. This role offers a unique opportunity to join a rapidly growing company at a defining moment in AI, with significant backing and a strong founding team. As a Software Engineer specializing in RL Environments, you will design simulations, data, and evaluations that directly influence how frontier models learn, working hands-on with research teams at top AI labs to experiment with environment design, pilot novel data creation strategies, and develop metrics for model improvement. Your work will feed directly into large-scale model training runs, allowing you to go from hypothesis to live experiment quickly.
Responsibilities
- Construct simulated worlds and explore data shapes that expose meaningful model failure modes across domains like finance, code, and enterprise workflows.
- Build and refine evaluation rubrics and reward signals for RLHF and RLVR training pipelines.
- Analyze agent-produced trajectories and run experiments to improve different model capabilities.
- Develop quantitative frameworks for measuring dataset quality, diversity, and downstream impact on model alignment and capability.
- Create and manage both real world & synthetic data pipelines.
- Partner with lab research teams to translate their training objectives into concrete data and evaluation specifications.
- Partner with in-house researchers to run post-training experiments and scale training infrastructure.
Requirements
- Ability to design lightweight experiments, move fast, and extract actionable insights from messy results.
- Experience with Docker, or similar containerization tools, to design and monitor systems at scale.
- Strong familiarity with common reinforcement learning algorithms and methods, especially with respect to post-training LLMs.
Benefits
- Competitive salary and meaningful equity.
- Health Insurance: Medical, Vision, Dental
- 401(k) with Employer Match
- Daily UberEats Stipend
- Monthly Wellness Stipend