Engineering Manager, Deep Learning Inference
$184k - $357k • US, CA, Santa Clara • Full-time
Posted 1d ago
Remote Work Policy
On-site
Employment Type
Full-time
Categories
LLM Engineer
About the job
NVIDIA is seeking a Manager, Deep Learning Inference Software to lead a team focused on advancing AI model deployment. This role involves shaping software for sophisticated AI systems, including large language models and multimodal generative AI, accelerated on NVIDIA GPUs. The team develops and optimizes open-source frameworks like vLLM/SGLang and FlashInfer to make AI deployment scalable, efficient, and accessible, enabling real-time inference at all scales. The position offers a chance to work with forward-thinking individuals and provides opportunities for career advancement in a rapidly growing company.
Responsibilities
- Lead, mentor, and scale a high-performing engineering team focused on deep learning inference and GPU-accelerated software.
- Drive the strategy, roadmap, and execution of NVIDIA’s inference frameworks engineering, with a focus on Client AI.
- Partner with internal teams to deliver end-to-end optimized inference pipelines across NVIDIA accelerators.
- Oversee performance tuning, profiling, and optimization of large-scale models for LLM, multimodal, and generative AI applications.
- Guide engineers in adopting best practices for CUDA, Triton, CUTLASS, and multi-GPU communications.
- Represent the team in roadmap and planning discussions, ensuring alignment with NVIDIA’s broader AI and software strategies.
- Foster a culture of technical excellence, open collaboration, and continuous innovation.
Requirements
- MS, PhD, or equivalent experience in Computer Science, Electrical/Computer Engineering, or a related field.
- 6+ years of overall software development experience, including 3+ years in technical leadership or engineering management.
- Strong background in C/C++ software design and development; proficiency in Python is a plus.
- Hands-on experience with GPU programming (CUDA, Triton, CUTLASS) and performance optimization.
- Proven record of deploying or optimizing deep learning models in production environments.
- Experience leading teams using Agile or collaborative software development practices.
Benefits
- Highly competitive salaries
- Comprehensive benefits package
- Equity
- Opportunities for career advancement