Senior System Software Engineer - Dynamo-Triton Inference Server
$224k - $357k • US, CA, Santa Clara
Posted 1mo ago
About the job
We are seeking a Senior System Software Engineer to contribute to the development of the Dynamo-Triton Inference Server. This role involves building a highly-performant AI inference platform that makes designing and deploying new AI models easier and more accessible. You will be instrumental in developing world-class GPU-accelerated AI inference serving software, driving feature development, and fostering broad customer adoption. A key aspect of this role is the convergence of the Triton Inference Server and NVIDIA Dynamo stacks to create a unified, high-performance platform capable of serving both Large Language Model (LLM) and non-LLM workloads. You will also be an active participant in the open-source deep learning software engineering community, balancing objectives like building robust production-ready software, optimizing prediction throughput and latency, and adopting next-generation inference technologies.
Responsibilities
- Develop GPU-accelerated AI inference serving software.
- Contribute to feature development and drive customer adoption.
- Converge Triton Inference Server and NVIDIA Dynamo stacks into a unified inference platform.
- Serve both LLM and non-LLM workloads.
- Participate in the open-source deep learning software engineering community.
- Build robust software for production server or cloud environments.
- Optimize prediction throughput and latency.
- Develop and adopt next-generation inference technologies.
Requirements
- MS or PhD in Computer Science or relevant field, or equivalent experience.
- 12+ years of professional experience in deep learning software.
- Excellent Rust and C++ skills.
- Familiarity with Python.
- Strong programming and software design skills, including debugging, performance analysis, and test design.
- Experience with high-scale distributed systems and ML systems.
- Strong communication skills.
- Ability to work in a fast-paced, agile team environment.
Benefits
- Equity
- Benefits