TRT-LLM Jobs

NVIDIA is seeking a motivated Deep Learning engineer to integrate advanced communication technologies into AI stacks, including PyTorch, TRT-LLM, vLLM, and SGLang. You will work with the team that created communication libraries like NCCL and NVSHMEM, and technology like GPUDirect, to scale Deep Learning and HPC applications. Your work will directly impact AI application performance by improving communication between GPUs, making AI toolkits easier for the community to use. This role offers an outstanding opportunity for someone with an AI background to advance the state of the art in this space.

$184k - $357k

NVIDIA is seeking a Senior Deep Learning Software Engineer to join their Algorithmic Model Optimization Team. This role focuses on optimizing generative AI models, including LLMs and diffusion models, for maximum inference efficiency. The team utilizes techniques like neural architecture search, pruning, sparsity, and quantization, and develops an innovative software platform for automated deployment. The engineer will contribute to applied research and the development of this software, which is used both internally at NVIDIA and externally by other AI development teams. This is an opportunity to push the boundaries of inference efficiency and large-scale automated deployment, working with high-level frameworks and improving high-performance kernel implementations.

$184k - $357k

We are seeking a Senior System Software Engineer to contribute to the development of the Dynamo-Triton Inference Server. This role involves building a highly-performant AI inference platform that makes designing and deploying new AI models easier and more accessible. You will be instrumental in developing world-class GPU-accelerated AI inference serving software, driving feature development, and fostering broad customer adoption. A key aspect of this role is the convergence of the Triton Inference Server and NVIDIA Dynamo stacks to create a unified, high-performance platform capable of serving both Large Language Model (LLM) and non-LLM workloads. You will also be an active participant in the open-source deep learning software engineering community, balancing objectives like building robust production-ready software, optimizing prediction throughput and latency, and adopting next-generation inference technologies.

$224k - $357k

NVIDIA is seeking a motivated Deep Learning engineer to integrate advanced CUDA features and Distributed Runtime technologies into AI stacks, including frameworks like PyTorch, TRT-LLM, vLLM, SGLang, and JAX. You will join the team responsible for core CUDA features and runtimes that scale Deep Learning and HPC applications, addressing diverse multi-GPU demands from large-scale training to low-latency inference. Your work will enhance the productivity and performance of AI applications, accelerating community adoption through AI toolkits. This role offers a significant opportunity to advance the state of the art in AI technologies.

$184k - $357k