Research Scientist / Engineer – Training Infrastructure
$195k - $395k • Remote • Redwood City, CA • FullTime
Posted 9d ago
Job Location
Redwood City, CA
Tech Stack
Remote Work Policy
Fully remote
Employment Type
FullTime
Categories
AI Infrastructure Engineer
About the job
Luma is seeking a Research Scientist / Engineer to build and optimize the distributed systems that train large-scale multimodal models across thousands of GPUs. This role focuses on advanced parallelism, training stability, and utilization across massive clusters, requiring deep expertise in distributed PyTorch and CUDA. The goal is to provide researchers with reliable, efficient, and scalable infrastructure for innovation.
Responsibilities
- Design, implement, and optimize efficient distributed training systems for large-scale models.
- Research and implement advanced parallelization techniques (FSDP, Tensor Parallel, Pipeline Parallel, Expert Parallel).
- Build monitoring, visualization, and debugging tools for large-scale training runs.
- Optimize training stability, convergence, and resource utilization across massive GPU clusters.
Requirements
- Extensive distributed PyTorch training and parallelisms in foundation-model training.
- Deep understanding of GPU clusters, networking, and storage systems.
- Familiarity with communication libraries (NCCL, MPI) and distributed-system optimization.
- Experience with FSDP and multi-node training at scale.