Code Data Annotation Quality Specialist
Paris • Full-time
Posted 3mo ago
About the job
Mistral is seeking highly motivated Data Quality Specialists to join our Human Data Annotation team. This hybrid role involves reviewing and auditing code annotations against rubrics to ensure high-quality data for AI model training and evaluation. You will also be responsible for building, maintaining, and troubleshooting the internal tooling that annotators use daily. This position offers the opportunity to collaborate closely with annotators, technical program managers, and engineering stakeholders, contributing to the refinement of guidelines and processes that shape data production.
Responsibilities
- Generate and validate high-quality data annotations based on guidelines and feedback for AI model development and evaluation.
- Identify and report systemic issues, edge cases, and guideline gaps to annotation operations and technical stakeholders.
- Produce annotations to model the expected quality bar.
- Build and maintain internal tools and automation for annotator workflows, including visualization dashboards and batch configuration scripts.
- Troubleshoot environment, tooling, and CLI/git issues for annotators, coordinating with IT and engineering as needed.
Requirements
- Degree in computer science, engineering, or a related field, or 2-5 years of professional experience in software engineering, technical support, or tool development.
- Hands-on experience using code agents in a development workflow and interest in their evolution.
- Proficiency in at least one programming language (e.g., Python, JavaScript) with the ability to understand code across multiple languages.
- Ability to apply consistent judgment against a rubric and identify edge cases, ambiguities, or guideline gaps.
- Sustained focus and accuracy in detail-oriented, high-volume review work.
- Comfort working in a Unix-like terminal, including shell basics, package managers, environment setup, and git workflows.
- Ability to troubleshoot local development environment issues across common operating systems.
- Professional proficiency in English, with strong writing and comprehension skills.