Research Intern, Frontier Speculative Decoding (Summer 2027)
Remote • San Francisco
Posted 1d ago
Job Location
San Francisco
Tech Stack
Remote Work Policy
Fully remote
Categories
AI Research Engineer
About the job
The Speculative Decoding team at Together AI focuses on developing draft models and algorithms to accelerate large language model generation without altering outputs. Their work integrates model training, inference algorithms, and serving systems, optimizing all three for reduced cost and latency. As a research intern, you will explore open questions in speculative decoding, including new drafter architectures, verification schemes, knowledge distillation, and adaptive drafters. You will train and fine-tune draft models at scale on GPU clusters and evaluate your solutions against realistic serving workloads, such as long-context and agentic traffic.
Responsibilities
- Research and prototype new efficient speculative decoding algorithms and draft model architectures.
- Train, fine-tune, and distill draft models, and design experiments to validate hypotheses.
- Implement optimized versions for serving engines to demonstrate efficacy in real-world workloads.
- Communicate project plans, progress, and results to the team.
- Share findings through technical write-ups and blog posts, and contribute to productionizing successful ideas.
Requirements
- Currently pursuing a Master's or Ph.D. in Computer Science, Electrical Engineering, or a related field.
- Strong knowledge of Machine Learning and Deep Learning fundamentals.
- Hands-on experience training or fine-tuning large models with distributed training frameworks (e.g., Megatron-LM, FSDP).
- Familiarity with modern model architectures like Mixture-of-Experts and hybrid attention.