Research Intern, Frontier Speculative Decoding (Winter 2027)
Remote • San Francisco
Posted 1d ago
Job Location
San Francisco
Tech Stack
Remote Work Policy
Fully remote
Categories
AI Research Engineer
About the job
The Speculative Decoding team at Together AI focuses on developing draft models and algorithms to accelerate large language model generation without altering outputs. Our work spans model training, inference algorithms, and serving systems, aiming to reduce generation cost and latency. As a research intern, you will explore open questions in speculative decoding, including new drafter architectures, verification schemes, knowledge distillation, and adaptive drafters. You will train and fine-tune draft models at scale on GPU clusters and evaluate your ideas against realistic serving workloads, such as long-context and agentic traffic. Projects are designed to address key research questions in speculative decoding, leading to faster generation on frontier models, drafters that adapt to changing workloads, and robust evaluation under real traffic.
Responsibilities
- Research and prototype new efficient speculative decoding algorithms and draft model architectures.
- Train, fine-tune, and distill draft models, and design experiments to validate hypotheses.
- Implement optimized versions for serving engines to demonstrate efficacy in real-world workloads.
- Communicate project plans, progress, and results to the team.
- Share findings through technical write-ups and blog posts, and contribute to productionizing successful ideas.
Requirements
- Currently pursuing a Master's or Ph.D. degree in Computer Science, Electrical Engineering, or a related field.
- Strong knowledge of Machine Learning and Deep Learning fundamentals.
- Hands-on experience training or fine-tuning large models with distributed training frameworks (e.g., Megatron-LM, FSDP).
- Familiarity with modern model architectures, such as Mixture-of-Experts and hybrid attention.