Senior System Software Engineer, Agentic Inference - Dynamo
$224k - $431k • US, CA, Santa Clara
Posted 1d ago
About the job
NVIDIA is seeking a Senior System Software Engineer to join its GPU-accelerated deep learning software team, focusing on the Dynamo platform. This role involves developing open-source software for efficient Generative AI inference on GPUs, making AI model design and deployment more accessible. The team is building a fast-paced inference platform to support the growing AI revolution, enabling breakthroughs across various domains like image classification, speech recognition, and natural language processing.
Responsibilities
- Develop open-source software for inference of trained AI models on GPUs.
- Contribute to disaggregated serving for Dynamo-supported inference engines (vLLM, SGLang, TRT-LLM).
- Expand inference capabilities to support agentic workloads, including long-horizon reasoning, tool calling, and multi-turn execution.
- Innovate in inference-state management for long-running agents to reduce prompt processing, improve latency, and maximize GPU utilization.
- Build and evolve Dynamo’s distributed inference frontend across vLLM, SGLang, and TensorRT-LLM.
- Deliver day-0 support for new models, request parameters, API compatibility, and stateful response semantics.
- Balance objectives to build robust, scalable, high-performance software components.
- Prioritize features and capabilities in collaboration with team leads.
- Load-balance asynchronous requests across available resources.
- Optimize throughput under latency constraints.
- Integrate the latest open-source technology into the inference platform.
Requirements
- Master's or PhD degree or equivalent experience in Computer Science, Computer Engineering, or a related field.
- 10+ years of experience in Computer Science, Computer Engineering, or a related field.
- Ability to work in a fast-paced, agile team environment.
- Excellent Rust/Python programming and software design skills, including debugging, performance analysis, and test design.
- Understanding of modern LLM API semantics, including structured outputs, tool calling, reasoning controls, token accounting, context management, and multimodal inputs.
Benefits
- Equity
- Benefits