Member of Technical Staff, Post-Training

$200k - $350k • San Francisco, CA • FullTime

Posted 26d ago

Remote Work Policy

On-site

Employment Type

FullTime

Categories

Machine Learning Engineer

About the job

Handshake is seeking a Member of Technical Staff, Post-Training to help define and build a new organization focused on advancing the AI economy by organizing expert human knowledge. This role involves working directly with frontier labs on data, evaluation, and post-training challenges, building systems that transform expert knowledge into data and evaluations to improve frontier models. You will partner with researchers, domain experts, and customers to translate ambiguous post-training questions into experiments, evaluation frameworks, data pipelines, and products. As an early member, you will have significant influence over the technical direction, operating culture, and reusable systems built.

Responsibilities

  • Design post-training systems and methodologies for frontier models, including supervised fine-tuning, reinforcement learning, preference optimization, and reward modeling.
  • Translate research or partner needs into hypotheses, experiments, evaluation plans, and production-quality implementations.
  • Build and improve evaluation frameworks, benchmarks, training environments, data-processing pipelines, and quality-control systems.
  • Run iteration loops: prototype, evaluate, interpret results, and implement learnings into systems or products.
  • Partner with AI researchers and domain experts to develop high-signal data, feedback, and evaluation methods.
  • Identify repeatable patterns and productize them into reusable software and platforms.
  • Raise the technical bar through design judgment, clear communication, code quality, and mentorship.
  • Contribute to the field through benchmarks, open-source tools, research, and technical writing.

Requirements

  • 3+ years of demonstrated strength in post-training, fine-tuning, or model-evaluation work (e.g., RL, SFT, LoRA/PEFT, full fine-tuning, RLHF, DPO, PPO, reward modeling, training environments).
  • Strong Python skills and ability to write clean, efficient, scalable software.
  • Hands-on experience with modern ML tooling, particularly PyTorch and large-scale data, training, or evaluation workflows.
  • Sound experimental judgment: ability to form hypotheses, choose metrics, diagnose failures, and distinguish signal from noise.
  • Experience designing systems, including making tradeoffs around quality, scale, reliability, and reuse.
  • Comfort operating in an ambiguous, fast-moving environment with substantial ownership.
  • Collaborative, low-ego communication and ability to work effectively with diverse teams.
  • Experience building or operating large-scale ML training, inference, data, or evaluation systems.
  • Experience developing LLM/agent benchmarks, evaluation methodologies, annotation systems, or data-quality frameworks.
  • Research or applied work on reinforcement learning, alignment, model behavior, synthetic data, or human-in-the-loop systems.
  • Published research, meaningful open-source contributions, or evidence of technical leadership in ML systems or AI research.
  • Experience productizing research or customer work into robust, reusable platforms.

Benefits

  • Equity in a fast-growing company
  • 401(k) match
  • Competitive compensation
  • Financial coaching
  • Paid parental leave
  • Fertility benefits
  • Parental coaching
  • Medical, dental, and vision insurance
  • Mental health support
  • $500 wellness stipend
  • $2,000 learning stipend
  • Ongoing development
  • Commuting support
  • Free lunch
  • Gym in SF office
  • Flexible PTO
  • 15 holidays + 2 flex days
  • Team outings

About Handshake AI

Get new AI jobs in your inbox

A weekly digest of the newest AI engineering roles.

© 2026 AI Job Board. All rights reserved.