Research Engineer - Pretraining Data
$150k - $350k • Remote • San Francisco • FullTime
Posted 1d ago
Job Location
San Francisco
Tech Stack
Remote Work Policy
Fully remote
Employment Type
FullTime
Categories
AI Research Engineer
About the job
Prime Intellect is building the open superintelligence stack, providing infrastructure for AI labs. Their platform, Lab, unifies compute, environments, evaluations, and training for post-training at frontier scale. They are developing open-source models for long-horizon tasks and the platform their research team uses. The company has raised $150M from prominent investors and AI leaders. They are seeking individuals passionate about building at the intersection of frontier research, infrastructure, and go-to-market strategy for a nascent category.
This role focuses on building the data foundation for Prime Intellect's next generation of open models. You will identify and process valuable data sources, transforming them into training-ready corpora. Your work will directly influence both broad pretraining efforts and the development of specific capabilities like reasoning, coding, and long-context learning. You will collaborate closely with researchers to test and refine data strategies, ensuring the data effectively improves model performance.
Responsibilities
- Find, evaluate, and collect valuable training data from web, code, and technical sources.
- Build scalable pipelines for data extraction, cleaning, deduplication, and filtering.
- Develop and implement checks for data quality, coverage, and contamination.
- Collaborate with researchers on data mixtures and sampling strategies for pretraining and midtraining.
- Prepare and evaluate data for long-context learning, reasoning, and coding, including synthetic data.
- Conduct controlled data experiments and use model evaluations to guide scaling decisions.
- Ensure large-scale data processing is efficient, reproducible, and recoverable, with clear provenance and versioning.
Requirements
- Strong Python programming skills and experience building reliable data-processing or distributed systems.
- Experience with large datasets, managing tradeoffs in storage, compute, throughput, and reproducibility.
- Ability to diagnose data quality issues and translate research goals into measurable experiments.
- Comfort owning evolving technical problems end-to-end and collaborating with researchers.
- Experience with web crawling, document extraction, code-data collection, or large-scale corpus curation.
- Experience with deduplication, text classification, quality scoring, synthetic data, or training-data contamination analysis.
- Experience with data-mixture experiments, curriculum learning, or measuring dataset impact on model performance.
- Experience preparing datasets for language model pretraining, continued pretraining, or midtraining.
- Experience with distributed processing frameworks like Spark or Ray.
- Familiarity with multilingual, mathematical, scientific, or repository-level code data.
- Familiarity with tokenization, training data loaders, and large model training needs.
Benefits
- Cash Compensation Range of $150-350k
- Equity incentives
- Flexible work arrangements (remote or in-person)
- Visa sponsorship
- Relocation assistance
- Quarterly team off-sites
- Hackathons
- Conferences and learning opportunities