Member of Technical Staff - Data Ingestion Engineer
San Francisco, CA • FullTime
Posted 1mo ago
Remote Work Policy
On-site
Employment Type
FullTime
Categories
Applied AI Engineer
About the job
Reflection is a research lab dedicated to making intelligence open and accessible. As a Member of Technical Staff on the Data Team, you will be instrumental in building and operating the ingestion systems that transform large-scale data sources, such as the open web, into reliable corpora for training frontier AI models. This role involves owning the machinery for data acquisition, extraction, normalization, versioning, and delivery to pre-training pipelines. You will collaborate closely with world-class researchers, contributing to the critical link between data collection and its impact on model performance. This position is ideal for engineers who excel at building robust distributed systems while also enjoying experimentation, reasoning about data acquisition tradeoffs, and iterating quickly based on measurable outcomes.
Responsibilities
- Build and operate large-scale data ingestion systems for pre-training, including web crawling, extraction, and dataset delivery.
- Run experiments to evaluate crawling strategies, extraction methods, and ingestion tradeoffs.
- Analyze ingested data to identify gaps, redundancy, and areas for improvement.
- Build ingestion pipelines that scale reliably across large data campaigns.
- Develop specialized crawlers for high-priority data sources.
- Review code, debug production issues, and continuously improve ingestion infrastructure.
Requirements
- Experience building web crawling, data ingestion, or large-scale data acquisition systems using Ray, Beam, Spark, or similar technologies.
- Familiarity with how LLMs are trained and evaluated, and an intuition for what makes data useful for training.
- Comfortable working with very large datasets (multi-TB to PB scale) and building systems that are observable, testable, and maintainable.
- Comfortable designing experiments and using data to guide system improvements.
- Excellent communication skills, with the ability to explain system behavior and communicate tradeoffs clearly.
- Curious about how training data influences model capabilities and able to iterate quickly based on measurable downstream impact.
- Able to collaborate tightly across functions: researchers, infra, operations, and external partners.
- Enjoy working in a hybrid research–engineering role.
Benefits
- Top-tier compensation: Salary and equity structured to recognize and retain talent globally.
- Stock options.
- Comprehensive medical, dental, vision, and life insurance.
- Annual wellness allowance.
- Lunch and dinner provided in the office daily.
- 22 weeks paid parental leave for all new parents.
- Unlimited paid time off in the U.S.
- 30 days paid time off in the U.K.
- Sponsorship support for visas and long-term immigration pathways.
- Regular off-sites, happy hours, and team celebrations.