Research Engineer
$200k - $240k • Remote • San Francisco • FullTime
Posted 5h ago
Job Location
San Francisco
Tech Stack
Remote Work Policy
Fully remote
Employment Type
FullTime
Categories
AI Research Engineer
About the job
Outset is building AI-powered user interviews, creating digital twins that represent specific real people and respond to research questions as those individuals would. The Digital Twins Team focuses on measuring and improving how faithfully these AI systems reproduce human responses and behavior. As a Research Engineer, you will report to the Head Scientist and be responsible for building and maintaining the systems that enable the team's scientists to conduct their studies correctly, reproducibly, and at scale. Your role involves ensuring experiments execute as designed, results are reproducible, and research code stays aligned with the shipped product, supporting scientists in a computational research setting with hands-on experience in LLM-based systems.
Responsibilities
- Own and manage the evaluation infrastructure for testing modeling approaches, enabling quick setup, concurrent execution, and monitoring of new tests.
- Implement measurement pipelines precisely as specified by scientists, ensuring deterministic and traceable outputs.
- Maintain versioned evaluation datasets, including held-out sets not used for training.
- Build and maintain data pipelines for human-participant data, external datasets, and model training data, ensuring linkage of participant records across sources.
- Keep research code synchronized with the production product for prompt shipping of validated improvements.
- Detect and prevent threats to the validity of evaluations.
- Track compute and API spend across experiments and analyze their impact on production costs.
- Manage code repositories, conduct code reviews, maintain documentation, and ensure the reproducibility of the team's coding workflow.
- Identify and propose improvements for correctness and efficiency in implementation choices.
Requirements
- Several years of software engineering experience, including supporting researchers in a lab or comparable setting.
- Experience translating specific scientific requirements and experiment designs into code.
- Experience building and operating large-scale data pipelines on human behavior data, ensuring adherence to researcher specifications.
- Strong Python skills, including the ability to read and audit code from agentic coding tools.
- Solid fundamentals of computing, including file systems, software libraries, databases, SQL, tabular data, and Git-based workflows.
- Experience with cloud infrastructure and running many jobs concurrently.
- Working understanding of machine learning, including transformer-based language models, model API usage, and fine-tuning concepts.
- A habit of writing deterministic, reproducible, cost-aware, and well-documented code.