Member of Technical Staff - Data Quality Engineer (Pre-training)
San Francisco, CA • FullTime
Posted 6mo ago
Remote Work Policy
On-site
Employment Type
FullTime
Categories
Applied AI Engineer
About the job
Reflection is a research lab dedicated to making intelligence open and accessible. As a member of the Data Team, you will ensure the data used to train our models meets high standards for quality, reliability, and impact. You will work closely with world-class researchers on pre-training teams to translate abstract notions of "good data" into measurable standards for large-scale data campaigns. This role requires a blend of strong engineering fundamentals and a deep curiosity about data quality's influence on model performance.
Responsibilities
- Own upstream data quality for LLM pre-training across languages and modalities.
- Partner with research and pre-training teams to define measurable quality signals and provide feedback to data vendors.
- Design, validate, and scale automated QA methods for measuring data quality in large campaigns.
- Build reusable QA pipelines to deliver high-quality data for model training.
- Monitor and report on data quality trends, driving continuous improvement of standards and processes.
Requirements
- Strong engineering fundamentals with experience in data pipelines, QA systems, or evaluation workflows for pre-training data.
- Detail-oriented with an analytical mindset to identify data quality issues.
- Solid understanding of how data quality impacts pre-training and ability to translate concerns into actionable signals.
- Experience designing and validating automated quality checks (rule-based, statistical, or model-assisted).
- Comfortable working autonomously, owning problems end-to-end, and collaborating with researchers and engineers.
- Proficiency in Python for building ML/LLM workflows and writing scalable code.
- Experience with large datasets and automated evaluation or quality-checking systems.
- Familiarity with LLM workings, training, and evaluation.
- Excellent communication skills for articulating complex technical concepts.
Benefits
- Top-tier compensation (salary and equity).
- Stock options.
- Comprehensive health, dental, vision, and life insurance.
- Annual wellness allowance.
- Provided lunch and dinner in the office.
- 22 weeks paid parental leave.
- Unlimited paid time off (US) / 30 days paid time off (UK).
- Visa sponsorship support.
- Team building events (off-sites, happy hours, celebrations).