Senior Inference Engineer, GPU Kernel Optimization
$184k - $288k • Remote • US, CA, Santa Clara
Posted 1d ago
About the job
Our LLM Inference Performance Analysis and Optimization team is seeking a Senior Inference Engineer specializing in GPU Kernel Optimization. This role is crucial for maximizing the performance of LLM inference operations by developing advanced infrastructure and tooling. You will build silicon-measured kernel benchmarking systems, model-level performance projection tools, and agentic optimization systems that enhance GPU kernels at the assembly layer. The position involves close collaboration with compiler, kernel, hardware, and framework teams across NVIDIA to identify and address performance bottlenecks, driving measurable improvements in LLM inference.
Responsibilities
- Develop and implement GPU kernel microbenchmarking to measure competing kernel implementations with real-silicon fidelity.
- Conduct end-to-end model performance analysis, linking performance data to model serving economics and identifying optimization opportunities.
- Create optimization policies for production inference deployments.
- Apply AI-driven analysis for agentic kernel optimization, diagnosing performance gaps and exploring optimization strategies.
- Validate optimization findings through rigorous silicon measurements.
- Collaborate with compiler, hardware, kernel, and framework teams to deliver upstream improvements and production-grade performance gains.
Requirements
- Master's or PhD in Computer Science, Computer Engineering, or related field, or equivalent experience.
- 6+ years of relevant industry experience.
- Experience building or directing agentic AI systems (e.g., code generation, automated optimization, multi-step reasoning).
- Strong Python and C++ skills with proven software engineering fundamentals.
- Hands-on GPU profiling experience with CUPTI, NSYS, and NCU, with the ability to attribute bottlenecks.
- Direct experience with LLM inference frameworks like TRT-LLM, SGLang, or vLLM.
- Understanding of how kernel selection impacts model throughput and latency.
- Working knowledge of GPU kernel optimization (CUDA, CUTLASS, Triton, or equivalent).
- Ability to read PTX or SASS output.
Benefits
- Highly competitive salaries
- Comprehensive benefits package
- Equity