Senior Software Engineer, AI Infrastructure
$120k - $235k • United States, Washington, Redmond • Full-time
Posted 1mo ago
Job Location
United States, Washington, Redmond
Also open in United States, Washington, Redmond; United States, California, Mountain View
Tech Stack
Remote Work Policy
On-site
Employment Type
Full-time
Categories
AI Infrastructure Engineer
About the job
Our team builds the platform that manages the flow of AI traffic across Microsoft's global infrastructure, ensuring services like Azure OpenAI and Microsoft Copilot remain fast and reliable. As a Senior Software Engineer on this team, you will design, build, and operate services responsible for routing and prioritizing AI requests and matching them to available computing capacity. This role offers the opportunity to deepen your expertise in distributed systems at a global scale, gain insights into serving leading AI models, and develop as a technical leader and mentor.
Responsibilities
- Own the design of key platform components for routing AI traffic and managing computing capacity, ensuring goals for speed, reliability, scale, and cost are met.
- Write clear, secure, and well-tested code, providing constructive code reviews to foster team growth.
- Break down work into manageable milestones, utilizing feature flags and gradual rollouts for safe testing and rapid rollback.
- Participate in on-call rotations, enhance monitoring and dashboards, and improve troubleshooting to reduce incident frequency.
- Collaborate with product managers and partner teams to translate customer needs into technical requirements.
- Integrate security, privacy, and compliance into all stages of the development lifecycle.
- Automate testing and releases, and share knowledge and learnings with the team.
- Leverage AI tools throughout the software development lifecycle, taking ownership of AI-generated work.
Requirements
- Bachelor's Degree in Computer Science or related technical field and 4+ years of technical engineering experience building distributed systems with languages like C#, Java, or Go, or equivalent experience.
- Experience running distributed services on public cloud platforms (e.g., Azure, AWS, GCP).
- Experience with systems that route or schedule traffic, or allocate computing resources.
- Experience with Python, Docker, and Kubernetes.
- Experience with AI or ML infrastructure, such as model serving or GPU fleet management.
- Experience delivering projects with partner teams from design through launch and operation.
- Experience with on-call rotations for highly available services and improving monitoring to reduce incidents.
Benefits
- Annual base pay range: USD $119,800 - $234,700 (US) or $160,200 - $261,000 (SF Bay Area/NYC)
- Eligibility for other benefits and compensation
