Principal Engineer, Model Optimizations
$250k - $312k • Seattle • Full-time
Posted 23d ago
About the job
DigitalOcean is seeking a Principal Engineer to lead model optimization efforts for serving across their entire accelerator fleet, including NVIDIA and AMD hardware. This role is critical for bridging the gap between a model that simply runs and one that performs efficiently, directly impacting customer Service Level Objectives (SLOs) and GPU resource utilization. You will be responsible for the end-to-end model optimization process, encompassing quantization strategies, kernel selection and development, efficient execution of attention and Mixture of Experts (MoE) layers, speculative decoding, and parallelism layouts. This work involves adapting to rapidly evolving model architectures and diverse hardware vendor stacks, requiring the development of robust methodologies and tooling to ensure all models perform optimally on all GPUs, enabling rapid deployment of new models.
Responsibilities
- Set the technical strategy for model optimization across the GPU fleet, deciding which techniques to invest in, consume upstream, and how to evaluate them.
- Own quantization end-to-end, including calibration, accuracy budget management, and evaluation gates for techniques like FP8, FP4/MXFP4, INT8, AWQ, and GPTQ.
- Drive performance for modern architectures at the execution level, focusing on MoE routing, fused expert kernels, attention variants (MLA, GQA), long-context, and sliding-window attention.
- Lead speculative decoding efforts, including draft models, EAGLE/Medusa-class methods, and n-gram/lookahead approaches, tuning acceptance rates for optimal performance.
- Write and tune kernels using CUDA, Triton, CUTLASS, and their ROCm counterparts (HIP, Composable Kernel, hipBLASLt, AITER) when upstream solutions are insufficient.
- Establish repeatable methods for choosing parallelism layouts (TP, PP, EP, attention DP) based on model, GPU family, and traffic shape.
- Build benchmarking and regression infrastructure to ensure the trustworthiness of optimization claims, including traffic distributions, measurement of TTFT/ITL/throughput, and accuracy validation.
- Ensure AMD GPUs are a first-class target by driving upstream work in vLLM, SGLang, and TensorRT-LLM.
- Partner with NVIDIA and AMD engineering teams on pre-silicon enablement, early hardware access, and roadmap feedback.
- Set technical direction, mentor senior and staff engineers, and represent DigitalOcean in upstream communities, conferences, and customer technical discussions.
Requirements
- 12+ years of experience in performance-critical systems, with significant recent experience optimizing LLM inference in production.
- Deep understanding of GPU architecture and inference performance modeling, including memory bandwidth vs. compute bounds, arithmetic intensity, kernel launch overhead, and prefill vs. decode differences.
- Hands-on kernel-level experience on at least one vendor stack (CUDA/CUTLASS/Triton or ROCm/HIP/CK) with the ability and willingness to work across both.
- Practical expertise in quantization, including the judgment to assess accuracy implications beyond benchmark performance.
- Familiarity with the internals of at least one major serving engine (vLLM, SGLang, TensorRT-LLM) sufficient for upstream contributions.
- Strong Python and C++/CUDA skills, with proficiency in profiling tools like Nsight and rocprof.
- A measurement-first approach, backing claims with reproducible benchmarks and understanding inference benchmark limitations.
- Excellent written and verbal communication skills.
- Experience leading cross-functional efforts involving infrastructure, product, and customers.
Benefits
- Annual bonus plan eligibility
- Potential for new hire equity grant
- Hybrid work arrangement