Member of Technical Staff, Data Analysis and Evaluation
Remote • London • FullTime
Posted 7mo ago
Remote Work Policy
Fully remote
Employment Type
FullTime
Categories
Applied AI Engineer
About the job
Cohere is seeking a Member of Technical Staff in Data Analysis and Evaluation to ensure the quality, reliability, and performance of our large language models (LLMs). This role involves designing and conducting data collection tasks, assessing dataset quality, and analyzing model robustness and generalisability. You will collaborate with researchers, engineers, and data annotators to drive data-driven decisions and enhance AI system effectiveness. The position requires expertise in statistics, experimental design, and machine learning to ensure high-quality data and reliable model performance across diverse scenarios, contributing to Cohere's mission of advancing AI.
Responsibilities
- Design and oversee data collection tasks, including supporting human annotators and ensuring data quality.
- Develop and apply statistical methods to evaluate dataset quality and reliability.
- Analyze and assess the generalisability and robustness of ML systems across diverse use cases.
- Collaborate with teams to improve dataset quality and model performance.
- Train and fine-tune large language models (LLMs) on distributed training infrastructures.
- Conduct experiments to evaluate model performance and identify areas for improvement.
Requirements
- Extremely strong software engineering skills.
- Strong expertise in designing and conducting data collection tasks, including working with human annotators.
- Strong statistical skills and experience evaluating scientific experiments related to data collection and model performance.
- Experience analyzing datasets for quality, biases, and suitability for training ML models.
- Hands-on experience training large language models (LLMs) on distributed training infrastructures.
- Familiarity with evaluating and improving the generalisability and robustness of ML systems.
- Proficiency in programming languages such as Python and ML frameworks (e.g., PyTorch, TensorFlow, JAX).
- Excellent communication skills for cross-functional collaboration and presenting findings.
- One or more publications at top-tier AI/ML venues.
Benefits
- Weekly lunch stipend of $75/£75 or equivalent.
- Full health and dental benefits, including a mental health budget.
- RRSP matching, 401K, Pension Scheme.
- 100% Parental Leave top-up for up to 6 months.
- Annual enrichment benefits (arts & culture, fitness/wellness, quality time, workspace improvement).
- Education & learning stipend for conferences, courses, and coaching.
- 6 weeks of paid vacation (30 working days).
- Budget for traveling to other offices if remote.
- Annual company offsite.
- Daily lunch program, snacks, and social events (for in-office).
- Co-working benefit (for remote employees not near an office).
- $500 home office stipend.