New: upload your resume and get matched to AI jobs. Free to start →

Senior Software Engineer- AI Frameworks

$120k - $235k • United States, Multiple Locations, Multiple Locations • Full-time

Posted 10d ago

Job Location

United States, Multiple Locations, Multiple Locations

Tech Stack

Remote Work Policy

On-site

Employment Type

Full-time

Categories

AI Infrastructure Engineer

About the job

The Microsoft AI Frameworks team develops software, performance systems, and engineering tools that enable state-of-the-art AI models to run reliably and efficiently at cloud scale. This role involves working across model architectures, frameworks, compilers, runtimes, libraries, observability, benchmarking, and hardware platforms. Senior Software Engineers partner with model developers, researchers, hardware teams, and production services to accelerate model onboarding, improve performance and reliability, reduce deployment time and hardware footprint, and turn performance insights into durable platform capabilities. This is a hands-on individual-contributor role for engineers who enjoy solving ambiguous, end-to-end systems problems, combining strong software engineering fundamentals with curiosity about AI workloads, disciplined measurement, and a willingness to work across organizational boundaries to deliver production impact.

Responsibilities

  • Design, implement, test, and operate production-quality components across AI frameworks, runtimes, benchmarking systems, performance tooling, and service integrations.
  • Benchmark, profile, debug, and optimize large language model training and inference workloads across GPUs and Microsoft hardware.
  • Build automation and observability to detect regressions, improve reproducibility, surface actionable insights, and accelerate model and hardware onboarding.
  • Drive scoped projects from problem definition through deployment, balancing delivery speed, maintainability, reliability, and measurable customer or capacity impact.
  • Partner with researchers, model teams, infrastructure owners, and hardware vendors to diagnose cross-stack issues and deliver production-ready solutions.
  • Contribute to technical design reviews, engineering standards, operational health, and mentoring of other engineers.

Requirements

  • Bachelor’s Degree in Computer Science or a related technical field and 4+ years of technical engineering experience coding in languages such as C++, or Python, or equivalent experience.
  • Expertise in GPU or equivalent accelerator programming and kernel optimization using CUDA, Triton, or comparable technologies.
  • Experience using profiling and benchmarking to deliver measurable performance improvements, with an understanding of memory hierarchy, parallel execution, synchronization, and hardware utilization.
  • Familiarity with vLLM, SGLang, or equivalent inference frameworks, including their model execution, scheduling, batching, and KV-cache mechanisms.
  • Demonstrated technical ownership and cross-team collaboration, including independently delivering complex engineering changes into production.
  • Ability to use AI-assisted development tools effectively while maintaining sound engineering judgment and accountability for correctness, performance, and maintainability.

Benefits

  • Base pay range: USD $119,800 - $234,700 per year (US)
  • Base pay range: USD $160,200 - $261,000 per year (San Francisco Bay area and New York City metropolitan area)
  • Certain roles may be eligible for benefits and other compensation.