Research Engineer, Frontier Speculative Decoding
$190k - $270k • Remote • San Francisco, New York City
Posted 1mo ago
Job Location
San Francisco, New York City
Tech Stack
Remote Work Policy
Fully remote
Categories
AI Research Engineer
About the job
Together AI is building the Inference Platform that powers the world's most advanced generative AI models. This role will serve as a critical bridge between cutting-edge research and real-world applications, focusing on translating internal model training research into production-ready deployments for customers. The work involves a deep commitment to data-centric development, meticulous hyperparameter tuning, and rigorous checkpoint evaluation. You will transform general-purpose models into highly performant, specialized tools by fine-tuning them on customer-specific data and internal datasets, working with dedicated GPU clusters rather than training foundation models from scratch.
Responsibilities
- Design and iterate on novel speculator algorithms, combining architectural innovations with curated data to optimize accuracy-efficiency tradeoffs.
- Act as the critical link between raw data and production-ready models, ensuring your work directly impacts customer success.
- Collaborate directly with customers to understand their needs and integrate your work into the production platform with core inference and Applied ML research teams.
- Take on and solve challenging problems with deep technical ownership.
Requirements
- Genuine passion for data curation and processing with meticulous attention to detail.
- Demonstrated ability to perform effective hyperparameter searches and understand tuning trade-offs.
- Experience working with and building on existing training codebases.
- Strong attention to detail in evaluating model checkpoints for quality, performance, and reliability.
- Proficiency in Python and PyTorch.
- Familiarity with SLURM and/or Kubernetes clusters and high-performance computing job management.
- Familiarity with modern LLMs and generative models.
- Basic understanding of distributed training frameworks (e.g., FSDP, DeepSpeed).
- Bachelor's, Master's degree, or Ph.D. in Computer Science, Computer Engineering, or related field, or equivalent practical experience.
Benefits
- Competitive compensation
- Startup equity
- Health insurance
- Other competitive benefits