Research Engineer, Takeoff Intel
Remote • Remote-Friendly (Travel Required) | San Francisco, CA
Posted 6d ago
About the job
Anthropic is dedicated to creating AI systems that are reliable, interpretable, and steerable, aiming for AI to be safe and beneficial for users and society. The Takeoff Intel team specifically focuses on measuring the increasing delegation of AI development to AI systems themselves. This involves designing evaluations of AI R&D capabilities, building internal telemetry to track AI-assisted model development, and developing quantitative methods to understand AI capability growth. The team's work informs internal research priorities and safety planning, as well as external reporting on AI progress. As a Research Engineer on this team, you will build and operate the evaluation and measurement instruments essential for this research, working across infrastructure, data processing, and analysis tooling with a focus on delivering impactful prototypes.
Responsibilities
- Design, build, and run capability evaluations and measurement instruments at scale.
- Build data and analysis pipelines to transform model outputs and telemetry into reliable metrics.
- Rapidly prototype new instruments, validate them, and determine which to implement.
- Review and supervise AI-generated code as part of the regular workflow.
- Collaborate with research scientists and partner teams to define key measurement areas.
- Contribute to internal documentation and public reporting.
Requirements
- Shipped an evaluation, data product, or research library from start to finish.
- Ability to prototype quickly and discard code when necessary.
- Experience handling large volumes of messy data without over-engineering.
- Experience running experiments on large language models.
- Ability to work from vague research questions rather than detailed specifications.
- Clear communication of results and close collaboration with researchers.
Benefits
- Annual compensation range: $350,000 - $850,000 USD