Member of Technical Staff, Post-Training
$200k - $350k • San Francisco, CA • FullTime
Posted 26d ago
Job Location
San Francisco, CA
Tech Stack
Remote Work Policy
On-site
Employment Type
FullTime
Categories
Machine Learning Engineer
About the job
Handshake is seeking a Member of Technical Staff, Post-Training to help define and build a new organization focused on advancing the AI economy by organizing expert human knowledge. This role involves working directly with frontier labs on data, evaluation, and post-training challenges, building systems that transform expert knowledge into data and evaluations to improve frontier models. You will partner with researchers, domain experts, and customers to translate ambiguous post-training questions into experiments, evaluation frameworks, data pipelines, and products. As an early member, you will have significant influence over the technical direction, operating culture, and reusable systems built.
Responsibilities
- Design post-training systems and methodologies for frontier models, including supervised fine-tuning, reinforcement learning, preference optimization, and reward modeling.
- Translate research or partner needs into hypotheses, experiments, evaluation plans, and production-quality implementations.
- Build and improve evaluation frameworks, benchmarks, training environments, data-processing pipelines, and quality-control systems.
- Run iteration loops: prototype, evaluate, interpret results, and implement learnings into systems or products.
- Partner with AI researchers and domain experts to develop high-signal data, feedback, and evaluation methods.
- Identify repeatable patterns and productize them into reusable software and platforms.
- Raise the technical bar through design judgment, clear communication, code quality, and mentorship.
- Contribute to the field through benchmarks, open-source tools, research, and technical writing.
Requirements
- 3+ years of demonstrated strength in post-training, fine-tuning, or model-evaluation work (e.g., RL, SFT, LoRA/PEFT, full fine-tuning, RLHF, DPO, PPO, reward modeling, training environments).
- Strong Python skills and ability to write clean, efficient, scalable software.
- Hands-on experience with modern ML tooling, particularly PyTorch and large-scale data, training, or evaluation workflows.
- Sound experimental judgment: ability to form hypotheses, choose metrics, diagnose failures, and distinguish signal from noise.
- Experience designing systems, including making tradeoffs around quality, scale, reliability, and reuse.
- Comfort operating in an ambiguous, fast-moving environment with substantial ownership.
- Collaborative, low-ego communication and ability to work effectively with diverse teams.
- Experience building or operating large-scale ML training, inference, data, or evaluation systems.
- Experience developing LLM/agent benchmarks, evaluation methodologies, annotation systems, or data-quality frameworks.
- Research or applied work on reinforcement learning, alignment, model behavior, synthetic data, or human-in-the-loop systems.
- Published research, meaningful open-source contributions, or evidence of technical leadership in ML systems or AI research.
- Experience productizing research or customer work into robust, reusable platforms.
Benefits
- Equity in a fast-growing company
- 401(k) match
- Competitive compensation
- Financial coaching
- Paid parental leave
- Fertility benefits
- Parental coaching
- Medical, dental, and vision insurance
- Mental health support
- $500 wellness stipend
- $2,000 learning stipend
- Ongoing development
- Commuting support
- Free lunch
- Gym in SF office
- Flexible PTO
- 15 holidays + 2 flex days
- Team outings