Senior Software Engineer, GPU Code Generation for Machine Learning Inference
$152k - $288k • US, CA, Santa Clara
Posted 1d ago
About the job
NVIDIA's TensorRT team is seeking talented engineers to advance high-performance AI inference on NVIDIA GPUs. This role focuses on developing software that accelerates state-of-the-art generative AI models for local and enterprise deployments. You will tackle challenging problems in GPU code generation, just-in-time compilation, and inference optimization to deliver high performance across diverse models and hardware configurations. As a Senior Software Engineer, you will design and implement advanced GPU inference software, translating a deep understanding of GPU execution into efficient code generation and measurable improvements in generative AI inference.
Responsibilities
- Design and implement efficient GPU code generation techniques in TensorRT, balancing performance, compilation overhead, and startup latency.
- Develop compiler optimizations, including operator fusion, dynamic scheduling, and specialization for model shapes, data types, and GPU architectures.
- Profile workloads and analyze generated GPU code to identify bottlenecks and guide compiler and runtime improvements.
- Develop robust, maintainable software in C++ and Python for tooling, testing, and performance analysis.
- Collaborate with GPU architects, kernel developers, compiler engineers, and deep learning experts to guide hardware and software design.
Requirements
- BS, MS, or PhD in Computer Science, Computer Engineering, or related field, or equivalent experience.
- 4+ years of relevant software development experience, including contributions to a substantial codebase or project.
- Strong proficiency in C++ and experience building performance-critical software.
- Understanding of GPU architecture and parallel programming concepts (memory hierarchies, thread execution, synchronization, resource utilization).
- Rigorous approach to performance analysis and debugging, with ability to validate numerical correctness and explain performance tradeoffs.
- Strong problem-solving and communication skills, with ability to work independently and collaborate across teams.
Benefits
- Equity
- Benefits