Research Scientist - Frontier Benchmarks
$200k - $375k • Remote • New York City, NY (Hybrid); San Francisco, CA (Hybrid); United States (Remote)
Posted 15d ago
Job Location
New York City, NY (Hybrid); San Francisco, CA (Hybrid); United States (Remote)
Tech Stack
Remote Work Policy
Fully remote
Categories
AI Research Engineer
About the job
Snorkel AI is seeking a Research Scientist to lead the design of next-generation benchmarks and datasets for frontier model evaluation. This role involves defining evaluation criteria, creating datasets to measure performance gaps, and developing training data to address them. You will collaborate with customers and academic partners to ensure datasets are grounded in real-world performance needs. The ideal candidate is passionate about shaping the future of frontier AI measurement within a dynamic, cross-functional startup environment.
Responsibilities
- Design state-of-the-art datasets for frontier model training and evaluation based on current performance and academic partnerships.
- Translate benchmark insights into compelling narratives for customer presentations, technical reports, and go-to-market materials.
- Collaborate with data operations, product, engineering, and strategy teams to inform the company roadmap with research findings.
- Stay current with LLM evaluation research and integrate best practices into Snorkel's workflows.
- Represent Snorkel's research externally through publications, blog posts, conference talks, and customer engagements.
Requirements
- Strong research background in AI/ML evaluation, NLP, or related fields with a track record of rigorous experimental design.
- Exceptional communication skills, capable of presenting complex technical findings to diverse audiences.
- Comfortable operating in a fast-moving, cross-functional environment with ambiguous problem spaces.
- Genuine interest in GTM strategy, startup dynamics, and the commercial aspects of AI data services.
- Ph.D. in machine learning, NLP, or a related field, or equivalent industry/research lab experience.
Benefits
- Meaningful opportunities to shape priorities and initiatives.
- Influence key strategic decisions.
- Direct impact on ongoing success.
- Support for deepening technical expertise, exploring leadership, or learning new skills.
- Environment designed for growth, learning, and shared success.