Senior Deep Learning Frameworks CUDA Software Engineer
$184k - $357k • Remote • US, CA, Santa Clara
Posted 1mo ago
About the job
NVIDIA is seeking a motivated Deep Learning engineer to integrate advanced CUDA features and Distributed Runtime technologies into AI stacks, including frameworks like PyTorch, TRT-LLM, vLLM, SGLang, and JAX. You will join the team responsible for core CUDA features and runtimes that scale Deep Learning and HPC applications, addressing diverse multi-GPU demands from large-scale training to low-latency inference. Your work will enhance the productivity and performance of AI applications, accelerating community adoption through AI toolkits. This role offers a significant opportunity to advance the state of the art in AI technologies.
Responsibilities
- Integrate new CUDA features and Runtime abstractions into AI frameworks, from proof-of-concept to production.
- Analyze AI workloads and frameworks to identify innovation opportunities in lower-level stack components.
- Collaborate with teams working on the latest AI models.
- Drive improvements in the AI Compiler-Runtime interface for high-performance multi-GPU, multi-node solutions.
- Design fault-tolerant and elastic solutions for large-scale or dynamic AI workloads.
- Influence the roadmap of core CUDA to support next-generation DL frameworks.
- Collaborate with a dynamic, multi-timezone team.
- Work closely with AI researchers, architects, kernel and compiler authors, and CUDA driver experts.
- Develop exploratory tools and runtime systems to profile and accelerate new deep learning paradigms.
- Write clean, maintainable code, ensuring smooth transitions from prototypes to releases or products.
Requirements
- BS, MS, or PhD in Computer Science, Computer Engineering, Electrical Engineering, or related field (or equivalent experience).
- 8+ years of relevant industry or academic experience post-degree.
- Development experience with Deep Learning Frameworks (e.g., PyTorch, JAX) and Inference Engines (e.g., TRT-LLM, vLLM, SGLang).
- Rapid prototyping and development using Python, C++, CUDA, or related DSLs.
- Solid understanding of AI models, parallelisms, and/or compiler technologies (e.g., torch.compile).
- Experience with performance benchmarking on AI clusters and familiarity with performance profiler toolchains (e.g., PyTorch profiler, NVIDIA Nsight Systems).
- Understanding of HPC/AI communication concepts.
- Good grasp of computer system architecture, HW-SW interactions, and operating systems principles.
- Adaptability and passion for learning new frameworks and tools.
- Flexibility to work and communicate effectively across teams and timezones.
Benefits
- Equity
- Benefits