Senior AI Infrastructure Engineer Jobs

Nebius is building a full-stack AI cloud platform for the global AI economy, supporting developers and enterprises from data and model training through to production deployment. We are seeking a Senior Software Engineer to join our Nebius Serverless AI team, focusing on our GPU-native platform for deploying inference endpoints, batch jobs, and AI workloads without infrastructure management. This is a high-ownership role where you will define the technical direction, own the architecture of your contributions, and drive critical design decisions. The work involves end-to-end ownership, from runtime performance to customer-facing APIs, aiming to make our platform the fastest, cheapest, and most reliable for GPU workloads.

Salary not listed

Nebius is building a full-stack AI cloud platform for the global AI economy, supporting developers and enterprises from data and model training through to production deployment. Our Token Factory team is developing an inference platform for various foundation models, aiming to make them fast, reliable, and effortless to deploy at massive scale. We are seeking an engineer to ensure the flawless performance and graceful recovery of this platform under extreme load. This role involves owning the reliability, performance, and observability of the inference stack, from designing telemetry pipelines to tuning Kubernetes autoscalers and crafting resilient infrastructure. You will be instrumental in detecting, isolating, and remediating incidents quickly, and driving a post-mortem culture to prevent recurrence, all while meeting aggressive cost and reliability targets for scaling the platform.

Salary not listed

Nebius is building a full-stack AI cloud platform for the global AI economy, supporting developers and enterprises from data training to production deployment. Tavily, a part of Nebius, is developing the infrastructure layer for agentic web interaction at scale, powering Retrieval-Augmented Generation (RAG) and real-time reasoning in AI systems. This role is for a Senior Site Reliability Engineer who will manage Kubernetes clusters, own infrastructure as code, maintain CI/CD pipelines, and optimize real-time data pipelines. The engineer will also be responsible for building monitoring and alerting systems, debugging production issues, managing cloud costs, and collaborating closely with a small engineering team to own the entire infrastructure.

Salary not listed

Nebius is building a full-stack AI cloud platform to support developers and enterprises from data and model training through to production deployment. We are seeking a Senior HPC Cluster Engineer to join our GPU & InfiniBand team. This role is crucial for enhancing and optimizing the core components of our Cloud platform, with a specific focus on GPU computing, InfiniBand networks, and the KVM/QEMU stack. You will work with hardware virtualization and device emulation technologies to ensure high performance and security in multi-GPU, HPC environments. The position involves analyzing, troubleshooting, and improving infrastructure to support new hardware, fine-tuning system performance, and automating fault detection and resolution in a complex system.

Salary not listed

Together AI

We are seeking a software engineer to develop a Kubernetes-native control plane for managing our GPU inference fleet. This role involves designing a manifest-driven API that allows the inference team to declare their needs for clusters, model deployments, or capacity changes. Our controllers will handle the underlying reconciliation, provider/runtime selection, and lifecycle management, abstracting away the complexities of the serving stack, scheduler, and hardware pool. You will also build systems to optimize fleet efficiency, including defragmentation, rebalancing logic for consolidating workloads, and scheduling/bin-packing improvements to maximize GPU utilization without impacting latency. The primary goal is to decouple platform users from operational complexities while increasing usable capacity from existing hardware. This includes developing controllers, reconciliation loops, and self-service interfaces (API/CLI) for seamless decoupling, alongside event-driven systems for health monitoring, remediation, and utilization management without manual intervention. A product mindset is crucial, with experience building internal platforms or APIs for other engineering teams and a focus on developer experience and ownership of delivered software, including production support.

Salary not listed

Glean is seeking a Senior Site Reliability Engineer to enhance the reliability, scalability, and operational excellence of its production services. This role involves designing, building, and operating resilient cloud infrastructure, improving service reliability against defined SLOs, and assisting teams in responding to production issues through effective observability and incident management practices. You will provide technical expertise for system and architectural decisions and collaborate with engineers across the organization to deliver dependable systems. Glean operates in a hybrid cloud environment and faces complex distributed-systems challenges due to its rapid growth. Your work will involve using software engineering and SRE skills to automate tasks, build tools, and improve deployment, monitoring, and on-call workflows, ultimately helping Glean scale and provide a reliable customer experience.

$200k - $260k

Together AI

In this role, you will operate, scale, and optimize multi-petabyte storage systems purpose-built for the world’s largest AI training and inference workloads. You’ll manage and scale high-performance parallel filesystems and object stores, evaluate and integrate cutting-edge technologies, and solve complex engineering challenges of operating at extreme throughput, low-latency data paths, and massive cluster-scale storage operations. You will also build Kubernetes-native storage operators and self-service platforms that provide automated provisioning, strict multi-tenancy, performance isolation, and quota enforcement at cluster scale.

$250k - $330k

Together AI

We are seeking a software engineer to develop a Kubernetes-native control plane for managing our GPU inference fleet. This role involves designing a manifest-driven API that allows the inference team to declare their needs for clusters, model deployments, or capacity changes. Our controllers will handle the underlying reconciliation, provider/runtime selection, and lifecycle management, abstracting away the complexities of the serving stack, scheduler, and hardware. You will also build systems to optimize fleet efficiency, including defragmentation, rebalancing logic for consolidating workloads, and scheduling/bin-packing improvements to maximize GPU utilization without impacting latency. The primary goal is to decouple users of the platform from operational complexities while maximizing hardware capacity. This includes building controllers, reconciliation loops, and self-service APIs/CLIs, along with event-driven systems for health, remediation, and utilization, all designed to operate without human intervention. A product mindset is crucial, focusing on the developer experience of internal platforms and APIs, with a 'build it, own it' philosophy that includes production operation and support.

Salary not listed

Baseten

Baseten is seeking a Backend/Distributed Systems Engineer to join their Product Platform team. This role is central to Baseten's mission of powering mission-critical AI inference for leading AI companies. You will work on key areas such as commercialization, billing, partnerships, developer productivity, agentic development, and the observability platform. The position offers the opportunity to design and build scalable systems, own core platforms, and contribute to the company's growth at the forefront of AI development.

$165k - $330k

Databricks

Databricks is seeking a Senior Staff Software Engineer to work on the AI Runtime system, which powers the world's largest AI platform. This serverless product enables companies to train their own models using enterprise data, similar to how frontier labs operate. The role involves tackling complex technical challenges to advance AI Runtime's architecture, reliability, and developer experience for both humans and AI agents. You will contribute to shaping the long-term technical direction of AI training infrastructure, anticipating industry trends, and optimizing training efficiency and cost across various model architectures and GPU accelerators.

$229k - $343k

Scale AI

Scale GP is building the infrastructure that makes enterprise AI seamless. We are looking for a Senior Infrastructure Engineer to act as a primary technical lead, engineering the 'paved road' for our knowledge retrieval and inference engines. You won't just be managing resources; you’ll be defining the deployment standards for Agentic workflows at scale. Your mission is to bridge the gap between complex AI orchestration and world-class infrastructure, ensuring our platform remains the most reliable destination for enterprise agents. The ideal candidate thrives in a fast-paced environment, has a passion for both deep technical work and mentoring, and is capable of setting a long-term technical strategy for a critical domain while maintaining a strong, hands-on delivery focus. You will architect and implement solutions across multiple cloud providers (GCP, Azure, AWS) for customers in diverse, highly-regulated industries like healthcare, telecom, finance, and retail.

$216k - $270k

Vectara

Vectara is seeking a Senior Platform Software Engineer to join their team. This role focuses on developing and managing the core infrastructure for their enterprise RAG and Agentic AI Platform, ensuring high availability, scalability, and automation. You will work on Infrastructure as Code (IaC) and Helm charts, design and develop platform features, and contribute to the Forward Deployed Engineering function by supporting customer applications from POCs to production. The role involves collaborating with Field Engineering teams, deploying and optimizing ML/NLP workloads, and contributing to architecture and infrastructure decisions as the platform scales. You will also be part of an on-call rotation and contribute to improving system reliability and developer experience.

Salary not listed

Invisible is an Applied AI research company that builds platform infrastructure to enable AI in production at scale. We help companies unify data, orchestrate agents, and deploy workflow applications. Our platform supports clients like Grubhub and Thomson Reuters, creating a feedback loop between enterprise deployment and model training. We are recognized for our predictive analytics platform and are focused on making AI work effectively in real-world scenarios.

Salary not listed

Zencoder.ai is building and orchestrating AI agents designed to handle real-world tasks like coding, research, and operations. What began as developer tooling has evolved into a collaborative platform for humans and AI agents working together on knowledge-based tasks. This role focuses on 'AI harness engineering,' ensuring the fundamental systems like CI/CD, testing, infrastructure as code, observability, and secure delivery are robust enough for AI agents to operate productively. The goal is to enhance developer experience and platform engineering, making workflows smoother for both humans and AI, and to integrate security seamlessly into the development process, addressing the evolving threat landscape posed by AI agents.

Salary not listed

As a member of the Hardware Infrastructure EDA Compute team, you will optimize, scale, and support workload scheduling systems that directly impact design velocity and infrastructure efficiency. You will combine deep scheduler expertise (LSF, Slurm) with automation, CI/CD, and LLM-based tooling to keep the EDA and GPU-accelerated AI compute environment resilient and measurable.

Salary not listed

Relace

Relace is building a globally distributed inference cloud, competing on price, speed, and reliability. We process over 800 billion tokens daily and have experienced significant growth, backed by a $23 million Series A funding round. Our team is small, offering substantial responsibility and growth opportunities. We develop our own stack, running on Kubernetes across multiple clouds, where performance directly impacts business success. This role focuses on the lowest level of our stack, optimizing code that runs directly on GPUs to maximize work per second and dollar, shrink models for cost-efficiency without sacrificing quality, and eliminate GPU idle time.

Salary not listed

As a Senior Director of Software Engineering at JPMorgan Chase, you will lead multiple technical areas and manage departments responsible for delivering a unified AI infrastructure layer across on-premises environments, public cloud, and emerging accelerated-compute vendors. You will collaborate across AI/ML engineering, infrastructure, security, and vendor teams to ensure the firm remains at the forefront of AI platform capabilities and operational excellence. In this role, you will own training and experimentation on a Kubernetes-standardized platform, acting as an active design partner to guide architectural trade-offs and ensure designs translate into reliable, secure, and operable systems at enterprise scale.

Salary not listed

Lila Sciences is developing the infrastructure for autonomous science, creating AI systems that can design and execute experiments in physical labs across life sciences, chemistry, and materials science. Their platform is designed for reliability and scale, catering to external customers who demand security, auditability, and operational rigor. This role focuses on building foundational enterprise systems to support large pharmaceutical and research organizations, enhancing capabilities like multi-tenancy, identity management, granular permissions, audit infrastructure, and billing. The position involves pushing the boundaries with agentic AI-powered systems for self-service tenant management, intelligent customer observability, and automated compliance workflows, alongside developing an AI-native external API.

$192k - $288k

SambaNova

SambaNova is seeking an Inference Systems Performance Architect to lead the end-to-end performance strategy for their AI inference platform. This role involves defining and driving technical direction in workload capture, benchmarking, modeling, and simulation. You will be instrumental in understanding how requests move through the system, from tokenization to fabric communication, and how deployments meet customer service level objectives. The work focuses on creating reproducible representations of real and agentic traffic, developing what-if capabilities for future configurations, and tackling complex challenges in heterogeneous inference environments.

$245k - $325k

SambaNova

SambaNova is seeking a Senior Software Engineer to join the ML Infrastructure team. This role is central to our mission as an inference-first company, focusing on designing, building, and operating the production-grade inference infrastructure that powers SambaNova's serving stack on our Reconfigurable Dataflow Unit (RDU) architecture. You will transform state-of-the-art inference techniques into reliable, high-throughput, low-latency services exposed to customers through SambaStack and SambaCloud. This position involves end-to-end ownership of systems including request scheduling, advanced decoding algorithms, caching layers, API surfaces, and accuracy infrastructure, and requires close collaboration with ML, compiler, runtime, and product teams.

$200k - $275k

Cerebras

Cerebras Systems is developing a new generation of disaggregated AI inference systems that combine GPU-accelerated prefill with ultra-fast decode on the Cerebras Wafer-Scale Engine. We are seeking a Software Engineer to productionize and optimize our GPU serving stack. This role involves working across custom inference APIs, the vLLM serving runtime, the AMD ROCm software stack, and rack-scale AMD GPU infrastructure to ensure reliability, numerical correctness, observability, and exceptional performance. You will write production code, establish operational practices for a new accelerator fleet, and drive improvements in key performance metrics like time to first token, throughput, tail latency, and capacity efficiency. This is a hands-on position requiring deep debugging and optimization across application, runtime, distributed systems, and hardware layers.

Salary not listed

Pallet is building AI Agents to transform the physical economy, a massive global industry. We are a well-funded, high-growth startup with significant revenue growth and a mission to increase the efficiency of the global supply chain by automating manual workflows. Our platform provides end-to-end visibility and control, and our AI Agents integrate directly with customer systems to execute requests in real time. This role is for a Senior Member of Technical Staff, Infrastructure Engineer, focused on solving hard scaling and reliability problems to enable the next order of magnitude growth for Pallet's AI-powered supply chain solutions.

$200k - $265k

Anthropic

The Interpretability team at Anthropic is dedicated to understanding how modern language models work, believing that a deep, mechanistic understanding is crucial for making advanced AI systems safe and steerable. This role involves building and maintaining the specialized infrastructure that powers interpretability research, akin to performing 'neuroscience' on neural networks. The work spans the entire lifecycle of a production language model, from pretraining and inference to performance optimization, and directly contributes to safety audits on frontier models.

$315k - $560k

Glean is seeking a Backend/Infrastructure Engineer to build and evolve the Storage layer that powers a highly available, performant, secure, and cost-effective suite of storage options for all data at Glean. You will own the storage systems that handle Glean’s most sensitive data, from enterprise documents and permission hierarchies to user behavioral signals and real-time query path data. Your work will directly enable organizations at scale to discover and act on their critical knowledge, supporting millions of searches, chat interactions, and agent workflows. You’ll deliver robust uptime guarantees, autoscaling, and reliable alerting/monitoring for a key component that runs across multiple cloud providers. As part of the Storage team, you’ll be responsible for a critical piece of Glean’s infrastructure, working closely with several other teams to support their workloads. You’ll also work across the stack to abstract cloud differences, reduce human intervention through self-healing systems, and drive real business impact by enabling Glean’s ever-growing scale.

$200k - $300k