Human Data Architect
Remote • United States - Remote • FullTime
Posted 3mo ago
Job Location
United States - Remote
Tech Stack
Remote Work Policy
Fully remote
Employment Type
FullTime
Categories
Applied AI Engineer
About the job
Surge is building a platform that powers the most powerful AI models in partnership with leading companies. We believe the path to AGI involves embracing human intelligence and creativity in the data that shapes these systems. Our platform combines elite human expertise with cutting-edge tools for scalable oversight, from building rich RL environments to conducting rigorous evaluations.
As a Human Data Architect, you will design the systems that govern how human data is collected, structured, and delivered to train frontier AI models. This involves careful consideration of task design, evaluation methodology, and data pipeline architecture, with a focus on how upstream design choices impact downstream model behavior. You will work at the intersection of ML post-training and human data collection methodology, understanding the requirements for SFT examples, RLHF pipelines, and evaluation datasets, and building the systems to produce this data reliably at scale.
Responsibilities
- Architect end-to-end data workflows from task design through quality assurance and delivery, ensuring data is well-structured for its post-training purpose.
- Design collection structures and task frameworks to translate client model needs into scalable data generation processes.
- Build and monitor data quality frameworks to identify and resolve issues before they reach the client.
- Partner with engineering and product teams to enhance data pipelines, identifying structural or methodological improvements for throughput and quality.
- Translate patterns in data quality and project structure into actionable insights for project design, client deliverables, and product development.
Requirements
- Experience designing human data collection at scale in fields like computational cognitive science, HCI, or crowdsourcing research.
- Familiarity with ML post-training paradigms (SFT, RL) and an understanding of how data quality impacts model outcomes.
- Strong technical skills with the ability to work across data pipelines.
- Systems-level thinking to understand how task design choices affect training data.
- Excellent communication skills to explain technical concepts to diverse teams.