Research Engineer, Large-Scale Training
$200k - $290k • Remote • San Francisco
Posted 24d ago
Job Location
San Francisco
Tech Stack
Remote Work Policy
Fully remote
Categories
Machine Learning Engineer
About the job
The Model Shaping team at Together AI focuses on tailoring open foundation models for downstream applications, building services that help machine learning developers select and improve models with domain-specific data. We also develop novel methods for efficient model training and evaluation, drawing from machine learning, natural language processing, and ML systems.
As a Research Engineer on the Scaling Team, you will translate cutting-edge research in efficient foundation model training into robust, high-performance systems. You will optimize Together's training infrastructure, identify performance bottlenecks, and implement state-of-the-art techniques in production environments.
Your work will directly influence the fine-tuning experience for Together's customers. You will integrate newly released open-source models onto the platform, ensuring efficient and reliable training across diverse workloads. Collaborating with Research Scientists, you will build experimental infrastructure to accelerate research and deploy validated ideas at scale.
Responsibilities
- Design, implement, and optimize core components of large-scale training infrastructure.
- Integrate new model architectures, validate training correctness and convergence, and optimize performance for production fine-tuning.
- Profile distributed training workloads to identify and eliminate bottlenecks across compute, memory, and communication.
- Design and execute experiments to validate performance hypotheses and benchmark new approaches.
- Partner with Research Scientists to productionize novel training methods and contribute to publications and open-source releases.
- Rapidly enable support for newly released open-source foundation models on the platform.
- Build and maintain experimental infrastructure that accelerates research while ensuring production-quality reliability and scalability.
Requirements
- Demonstrated ability to independently take ambiguous performance or infrastructure problems from investigation through deployment.
- Strong programming skills in Python and PyTorch, with an emphasis on writing efficient, maintainable code.
- Hands-on experience training or fine-tuning large neural networks in multi-GPU or multi-node environments.
- Solid understanding of ML systems fundamentals, including GPU architecture, mixed-precision training, and distributed training paradigms.
- Strong communication skills and ability to collaborate effectively with researchers and engineers.
- Passion for staying current with advances in AI research and applying them to real-world systems.
- Excitement about translating cutting-edge research into production systems that deliver customer impact.