Research Engineer - Eval Platform
Paris • Full-time
Posted 1mo ago
Job Location
Paris
Tech Stack
Remote Work Policy
On-site
Employment Type
Full-time
Categories
AI Infrastructure Engineer
About the job
Mistral is seeking a Research Engineer to join their Eval Platform team. This role is crucial for ensuring the quality and reliability of AI models by building the infrastructure that science teams use to measure model performance. You will focus on making evaluations reproducible, comparable, and fast, directly impacting research decisions. The ideal candidate will have a strong software engineering background and a passion for AI, with a product-oriented mindset to serve the needs of researchers.
Responsibilities
- Build systems to ensure reproducible and comparable evaluation results over time.
- Run evaluations at scale across GPU clusters, including model serving and scoring.
- Develop APIs and dashboards for researchers to easily access, explore, and trust evaluation results.
- Implement mechanisms to detect and prevent misleading evaluation results.
- Support the evaluation of agentic, multi-turn, and tool-using models.
- Collaborate with researchers to translate new evaluation needs into robust, shared tooling.
Requirements
- Master's or PhD in Computer Science, or equivalent experience.
- 4+ years of experience building production-grade software, preferably in large-scale ML codebases or distributed systems.
- Excellent Python skills and strong software design principles (testing, code review, CI/CD).
- Experience running workloads on GPU clusters (e.g., Slurm, Kubernetes, Ray).
- Familiarity with LLM inference and evaluation.
- A product mindset, viewing researchers as primary users.
- Self-starter, collaborative, and low-ego attitude.
Benefits
- Comprehensive benefits package
- Healthcare coverage
- Parental leave
- Retirement plans
- Relocation support
- Wellness programs
- Meal and transportation allowances
