New: upload your resume and get matched to AI jobs. Free to start →

Research Engineer, Model Evaluation and Improvement

$136k - $167k • San Francisco, CA • Full-time

Posted 15d ago

About the job

We are rebuilding biotech for the AI era by developing an AI platform for biotech R&D. Scientists use our platform to design experiments, capture structured data, and run AI agents and models directly in their workflows. We are building an AI scientist for our customers and are committed to AI fluency as a core part of how we work. This role focuses on making frontier AI models better at science, addressing the gap in reasoning for real-world scientific problems. You will build the datasets, evaluations, and systems to close this gap, working with scientists to turn complex scientific work into rigorous tasks for models and partnering with AI labs to understand and improve model performance.

Responsibilities

  • Build datasets for evaluating and improving frontier models by transforming complex scientific data into high-quality tasks and environments for LLMs.
  • Analyze model failure modes through experiments on frontier models to identify areas for improvement.
  • Develop scalable data infrastructure and pipelines for curating, transforming, and validating scientific data for model evaluation and improvement.
  • Collaborate with frontier AI labs to develop and evaluate new methods for enhancing model performance on challenging scientific tasks.
  • Translate expert scientific judgment into problems and evaluation criteria to reliably distinguish strong model behavior.

Requirements

  • 2+ years of experience at the intersection of biology and AI, with a focus on evaluating and improving scientific models or LLMs for biological applications.
  • Experience building with LLMs, including an understanding of their capabilities and limitations, and how to design systems around them.
  • Curiosity and enthusiasm for frontier AI, with a drive to understand and advance the capabilities of rapidly evolving models.
  • Comfort working on ambiguous problems in a rapidly shifting technical landscape.
  • Collaborative mindset, with the ability to work effectively with engineers, scientists, and external research partners.
  • Desire to work in a fast-paced environment that encourages rapid experimentation and adaptability to shifting priorities.