LLM reasoning Jobs

We are seeking a Research Scientist to focus on Agentic Data & Benchmarking. This role involves designing and executing evaluations for agentic capabilities, including multi-step reasoning, tool use, and long-horizon planning. You will build and maintain robust evaluation harnesses to ensure reliable, large-scale benchmarking against model checkpoints, identifying regressions and monitoring model health. Additionally, you will conduct experiments to understand how various factors like prompting, sampling, and environment design influence agentic performance on both internal and public benchmarks. A key part of this role is diagnosing anomalous evaluation results during training, pinpointing the root cause (model, data, harness, or infrastructure), and communicating findings clearly. You will also be responsible for sourcing, generating, and curating high-quality agentic training data, including trajectories and tool-use traces, and designing and scaling RL environments and reward signals. Managing technical relationships with external data vendors and domain experts, evaluating data quality, and developing QA frameworks to ensure data and benchmark integrity are also crucial aspects of this position. You will contribute to technical reports, research publications, and open-source benchmarks and tooling, partnering with research and product teams to translate capability goals into measurable data and evaluation artifacts.

$150k - $450k