New: upload your resume and get matched to AI jobs. Free to start →

Principal Software Engineer - AI Infrastructure

$143k - $275k • United States, Washington, Redmond • Full-time

Posted 17d ago

Job Location

United States, Washington, Redmond

Also open in United States, Washington, Redmond; United States, Washington, Redmond

Tech Stack

Remote Work Policy

On-site

Employment Type

Full-time

Categories

AI Infrastructure Engineer

About the job

Help build the infrastructure that trains the next generation of agentic Artificial Intelligence (AI) models at Microsoft. You will work alongside engineers and researchers to advance the scientific methods and scalable infrastructure used to train Large Language Models (LLMs), Small Language Models (SLMs), and agentic models, spanning both proprietary and open-weight models and building on open-source training frameworks. As a Principal Software Engineer on this team, you will own the architecture and technical direction of major components of the training platform and drive them from ambiguous problem statements through production operation. You will keep the customer at the center of every decision, partnering directly with the researchers and product teams who rely on the platform to understand their needs, remove friction, and measure success by their outcomes.

Responsibilities

  • Own the architecture and technical direction of major components of the training platform for LLMs, SLMs, multimodal, code-specific, and open-weight models, ensuring scalability, reliability, security, and cost-efficiency.
  • Partner with customers, including researchers and product teams, to understand their training and evaluation needs, translate them into platform requirements, and measure success through customer outcomes.
  • Drive multi-quarter initiatives from proposal to production, managing milestones, dependencies, and delivery commitments with partner teams.
  • Build on and extend open-source training frameworks (e.g., PyTorch, DeepSpeed, Megatron-LM, Hugging Face libraries), adapting them for proprietary and open-weight models.
  • Design and build services that serve production traffic while meeting Microsoft security, privacy, Responsible AI, and compliance requirements.
  • Lead debugging and resolution of complex issues in distributed training jobs and production services, including on-call rotation and incident retrospectives.
  • Mentor engineers and raise the technical bar through design and code reviews, and ownership of the team's test strategy.
  • Champion AI-assisted engineering tools and practices across the software development lifecycle.

Requirements

  • Bachelor's Degree in Computer Science or related technical field AND 6+ years technical engineering experience with coding in languages including C, C++, C#, Java, JavaScript, or Python, OR equivalent experience.
  • Master's Degree in Computer Science or related technical field AND 8+ years technical engineering experience with coding in languages including C, C++, C#, Java, JavaScript, or Python, OR Bachelor's Degree in Computer Science or related technical field AND 12+ years technical engineering experience with coding in languages including C, C++, C#, Java, JavaScript, or Python, OR equivalent experience.
  • 5+ years designing, building, and operating large-scale distributed systems or cloud platforms with end-to-end ownership of production services.
  • Experience with Kubernetes-based containerized platforms on Azure, AWS, or Google Cloud.
  • Experience with control-plane or backend services managing compute, storage, and networking.
  • Experience with secure multi-tenant designs covering authentication, authorization, workload identity, quota management, and isolation.
  • 3+ years building machine learning training infrastructure, such as GPU cluster scheduling, data pipelines, and checkpointing and storage systems, using open-source training frameworks.