Research Scientist
$200k - $240k • Remote • San Francisco • FullTime
Posted 4h ago
Job Location
San Francisco
Tech Stack
Remote Work Policy
Fully remote
Employment Type
FullTime
Categories
AI Research Engineer
About the job
Outset is building AI systems, referred to as digital twins, that represent specific real people and can respond to research questions as those individuals would. These systems are used by global brands to gain deeper qualitative insights about their users at unprecedented speed and scale. The Digital Twins Team focuses on measuring and improving the faithfulness of these AI systems in reproducing human responses and behavior. The Research Scientist will work closely with the Head Scientist to define research questions, design studies, establish measurement criteria, interpret findings, and contribute to internal decision-making and potential academic publications. This role requires a strong background in computational social science, data science, survey methodology, human-computation interaction, and experiment design.
Responsibilities
- Co-design and conduct studies comparing AI-generated responses with human responses using qualitative and quantitative measures.
- Design, justify, and refine performance metrics for AI systems.
- Apply and adapt advanced statistical techniques for benchmarking and improving simulation performance on large-scale social data.
- Design evaluations and experiments with pre-specified analysis plans.
- Design and manage human rating studies, including rater training, coding rubrics, and inter-rater agreement.
- Design data collection from third-party sources and human participants, including longitudinal studies.
- Contribute to research on new modeling approaches, including evaluating fine-tuned models and model calibration.
- Specify analyses clearly for engineering implementation.
- Co-author research papers, preprints, and presentations.
Requirements
- PhD in computational social science, communication, HCI, sociology, political science, psychology, statistics, or a related field (completed or near completion).
- Peer-reviewed publications with you as a primary author.
- Proficiency in graduate-level statistical methodology common in social science, including understanding underlying assumptions.
- Experience analyzing large-scale telemetry and behavioral datasets, ensuring inference validity.
- Familiarity with machine learning approaches (classical, nonparametric, LLM-based), including transformer-based language models, fine-tuning, and model family differences.
- Understanding of fundamental computing concepts: file systems, software libraries, databases, SQL, and tabular data.
- Ability to work with and review code generated by agentic coding tools.