Member of Technical Staff, Post-Training (India)
Salary not listed • India • Full-time
Posted 21d ago
Remote Work Policy
On-site
Employment Type
Full-time
Categories
LLM Engineer
About the job
Handshake AI works with frontier labs on their most consequential data, evaluation, and post-training challenges, building systems that turn expert human knowledge into data and evaluations to improve frontier models. Handshake Labs is developing external AI products, research platforms, and customer-facing AI systems, evolving custom-built work into reusable products and platforms. This role is for a Member of Technical Staff, Post-Training, a broad, high-ownership position for researchers who build, requiring the ability to reason deeply about model improvement and translate that reasoning into reliable systems. You will partner with researchers, domain experts, and customers to transform ambiguous post-training questions into experiments, evaluation frameworks, data pipelines, and products, with significant influence over technical direction and culture.
Responsibilities
- Design post-training systems and methodologies for frontier models, including supervised fine-tuning, reinforcement learning, preference optimization, and reward modeling.
- Translate research or partner needs into hypotheses, experiments, evaluation plans, and production-quality implementations.
- Build and improve evaluation frameworks, benchmarks, training environments, data-processing pipelines, and quality-control systems.
- Run iteration loops: prototype, evaluate, interpret results, and implement learnings into systems or products.
- Partner with AI researchers and domain experts to develop high-signal data, feedback, and evaluation methods.
- Identify repeatable patterns and productize them into reusable software and platforms.
- Raise the technical bar through strong design judgment, clear communication, code quality, and mentorship.
- Contribute to the field through benchmarks, open-source tools, research, and technical writing.
Requirements
- 3+ years of demonstrated strength in post-training, fine-tuning, or model-evaluation work (e.g., RL, SFT, LoRA/PEFT, full fine-tuning, RLHF, DPO, PPO, reward modeling, training environments).
- Strong Python skills and ability to write clean, efficient, scalable software.
- Hands-on experience with modern ML tooling, particularly PyTorch and large-scale data, training, or evaluation workflows.
- Sound experimental judgment: ability to form hypotheses, choose metrics, diagnose failures, and distinguish signal from noise.
- Experience designing systems, including making tradeoffs around quality, scale, reliability, and reuse.
- Comfort operating in an ambiguous, fast-moving environment with substantial ownership.
- Collaborative, low-ego communication and ability to work effectively with diverse stakeholders.
Benefits
- Generous Equity Grant vested over 4 years
- Housing Bonus: 1.3 Lakhs spread throughout the first year
- Performance Bonus ranging between 10 - 100% of base
- Medical Insurance Coverage
- Food credit for every in person day