Senior Software Engineer - GPU Kernel Authoring & Optimization
$182k - $242k • Remote • Sunnyvale, CA / Bellevue, WA
Posted 5d ago
Job Location
Sunnyvale, CA / Bellevue, WA
Tech Stack
Remote Work Policy
Fully remote
Categories
Machine Learning Engineer
About the job
CoreWeave is seeking a Senior Engineer for its Benchmarking & Performance team, focusing on GPU kernel authoring and optimization. In this role, you will be responsible for writing, profiling, and tuning GPU kernels that are critical for large-scale model serving, aiming to maximize throughput and minimize latency. You will also contribute to industry-leading performance benchmarking publications. This position requires an individual who can lead designs, improve engineering standards, and deliver significant enhancements to latency, throughput, and reliability across the inference stack. You will collaborate with product, orchestration, and hardware teams to translate kernel-level optimizations into system-wide improvements and meet stringent Service Level Agreements (SLAs) at scale.
Responsibilities
- Author, profile, and optimize CUDA kernels for LLM inference, including GEMMs, attention, MoE routing, quantization, KV-cache, and fused epilogues.
- Optimize kernels for GPU hardware, leveraging tensor cores and tuning occupancy, memory coalescing, shared memory/register usage, and compute/data movement overlap.
- Use kernel-authoring DSLs and compilers for rapid prototyping and efficient kernel development.
- Conduct rigorous benchmarking, creating reproducible microbenchmarks and roofline analyses, and validating kernel performance gains in end-to-end model-serving stacks.
- Implement and maintain benchmarking workflows for MLPerf Inference and Training, including workload setup, cluster configuration, runbooks, and result validation.
- Lead design reviews and drive architectural decisions within the team, breaking down complex work into clear milestones.
- Mentor junior engineers and review cross-team designs to elevate coding and testing standards.
- Ensure reproducible and well-documented benchmarking and kernel-optimization processes.
Requirements
- 5+ years of experience in high-performance computing, GPU/accelerator software, or performance-critical systems.
- Hands-on CUDA experience, including writing and optimizing custom kernels and fluency with the CUDA programming and memory model.
- Deep understanding of GPU architecture and performance, including tensor cores, warp/occupancy tuning, memory hierarchy and bandwidth, NVLink/PCIe, and profiling tools like Nsight Compute/Systems.
- Strong proficiency in C++ and Python, with the ability to read and write low-level, performance-sensitive code.
- Familiarity with model-serving stacks such as vLLM, TensorRT-LLM, llm-d, and SGLang, and understanding of their dominant inference cost kernels.
- Excellent communication skills for collaborating with cross-functional teams and external partners.