New: upload your resume and get matched to AI jobs. Free to start →

Senior and/or Principal Software Engineer- AI GPU Performance

$143k - $275k • United States, Multiple Locations, Multiple Locations; United States, Washington, Redmond • Full-time

Posted 22d ago

Job Location

United States, Multiple Locations, Multiple Locations; United States, Washington, Redmond

Tech Stack

Remote Work Policy

On-site

Employment Type

Full-time

Categories

AI Infrastructure Engineer

About the job

The Microsoft AI Frameworks team is seeking experienced Senior and/or Principal Software Engineers to join their mission of enabling state-of-the-art AI models to run efficiently at cloud scale. This role focuses on optimizing the performance of Large Language Model (LLM) serving stacks, involving work across GPU kernels, distributed inference systems, and various hardware platforms. The successful candidate will be a hands-on engineer with strong systems engineering skills, deep performance expertise, and a disciplined approach to measurement and collaboration. You will be responsible for driving significant performance improvements, tackling complex cross-stack challenges, and creating reusable capabilities that advance serving performance for models like OpenAI and Microsoft AI.

Responsibilities

  • Develop and optimize GPU kernels and runtime components for LLM serving performance.
  • Profile workloads to improve latency, throughput, and hardware efficiency.
  • Integrate optimizations into production serving systems.
  • Collaborate across model, compiler, infrastructure, and hardware teams.
  • Drive cross-stack optimization across kernels, compilers, runtimes, and distributed serving.
  • Perform low-level optimization and resolve critical performance bottlenecks.
  • Turn performance innovations into reusable production capabilities.
  • Mentor senior engineers and raise engineering standards.

Requirements

  • Bachelor's Degree in Computer Science or related technical field and 4+ years of technical engineering experience coding in C++ or Python, or equivalent experience.
  • Expertise in GPU or equivalent accelerator programming and kernel optimization.
  • Familiarity with vLLM, SGLang, or equivalent LLM inference libraries.
  • Deep expertise in GPU or equivalent accelerator compilation pipelines and low-level architecture.
  • Expertise in LLM training and inference parallelism and acceleration.
  • Demonstrated cross-team collaboration and technical ownership.
  • Ability to use AI-assisted development tools efficiently and validate their output.

Benefits

  • Base pay range for IC5: USD $142,800 - $274,800 (US) or $188,000 - $304,200 (SF Bay Area/NYC)
  • Base pay range for IC4: USD $119,800 - $234,700 (US) or $160,200 - $261,000 (SF Bay Area/NYC)
  • Eligibility for other benefits and compensation