AI Infrastructure Engineer Jobs

Lila Sciences is seeking an Associate Director/Director, Inference Engineering to lead the team responsible for the company's model serving platform. This role involves defining how models are served at scale, balancing key performance metrics like latency, throughput, quality, and cost. The position acts as a technical authority, bridging the gap between research and production-grade serving capabilities, and encompasses people leadership, service ownership, cross-functional collaboration, and deep technical expertise in modern LLM serving.

$204k - $348k

Baseten

Baseten powers mission-critical inference for leading AI companies, enabling them to bring cutting-edge models into production. We are building the platform engineers use to ship AI products, recently securing $1.5B in Series F funding. The Agent Infra team is responsible for the core virtualization layer that powers our agent sandboxes. This layer provides each agent with its own isolated microVM, capable of running code, calling tools, and maintaining state across long tasks. We are building a cloud infrastructure from the ground up, focusing on multi-tenancy and deep hardware integration.

$165k - $330k

Turing

Enterprise AI is building the platform that enables reliable deployment of AI systems across enterprise environments. As a Platform Engineer, you will build and operate the core of Turing’s Enterprise AI platform, including platform services, the model gateway, multi-tenant infrastructure, CI/CD, infrastructure as code, and observability. The central challenge is creating repeatable deployments across environments Turing may not control, including client cloud accounts, dedicated instances, and sovereign installations. This role requires hands-on software engineering and operational ownership, with a focus on building production software and contributing deeply to platform services.

Salary not listed

Nebius is building a full-stack AI cloud platform to support developers and enterprises from data and model training through to production deployment. We are seeking a highly skilled ML/AI Engineer to lead and support benchmarking of GPU platforms for machine learning and AI workloads. You will play a critical role in evaluating the performance of GPU-based hardware for various deep learning and AI frameworks, enabling data-driven decisions for platform optimisation and next-generation hardware development. This role involves working closely with hardware and development teams to profile and analyse GPU performance, evaluate and compare GPU performance across different platforms, and debug and optimise ML workloads to run efficiently on GPU hardware.

Salary not listed

Nebius is building a full-stack AI cloud platform to support developers and enterprises in AI/ML infrastructure. This senior systems engineering role focuses on diagnosing and resolving complex hardware and platform failures in high-performance AI infrastructure. The engineer will work across Linux, GPU servers, PCIe, firmware, power, and cooling to determine the root cause of issues, developing new troubleshooting procedures for ambiguous problems. The role requires strong ownership, curiosity, and the ability to turn ambiguous problems into evidence-based conclusions.

$180k - $224k

Nebius is building a full-stack AI cloud platform to support developers and enterprises from data and model training through to production deployment. We are seeking a GPU Cluster Architect to drive the design of our next-generation AI infrastructure. In this high-impact, hands-on role, you will make end-to-end architectural decisions across compute, networking, and storage, ensuring our platforms can meet the massive scale, performance, and reliability requirements of modern AI workloads. You will define how tens of thousands of GPUs are interconnected, cooled down, powered, and optimized across multiple data center sites.

$184k - $318k

Nebius is building a full-stack AI cloud platform for the global AI economy, supporting developers and enterprises from data and model training through to production deployment. As a Forward Deployed Engineer, you will work alongside Physical AI customers to understand their workloads and production requirements, translating these into actionable insights for our core infrastructure products. You will also investigate and prototype new product opportunities based on unmet customer needs, helping to shape the future of Nebius' offerings. This hands-on role involves writing code, deploying and benchmarking models, building agents, and troubleshooting infrastructure in close collaboration with customers, NVIDIA, Product, and Engineering teams.

Salary not listed

Nebius is building a full-stack AI cloud platform for the global AI economy, supporting developers and enterprises from data and model training through to production deployment. We are seeking a Staff Software Engineer to focus on the indexing and data processing layer of a novel search engine designed for AI agents. This role involves building systems to ingest, process, and organize massive datasets into efficient, queryable structures, operating on offline and nearline pipelines to ensure data freshness and accessibility for retrieval systems. The position requires designing and implementing large-scale indexing systems capable of handling tens of gigabytes per second across continuously evolving datasets, with a strong emphasis on throughput, scalability, and correctness.

Salary not listed

Scale AI

Scale AI is pioneering the next era of enterprise AI, providing cutting-edge solutions that transform workflows and automate complex processes for large enterprises. The Scale Generative AI Platform (SGP) offers foundational services and APIs for seamless AI integration at production scale. This role focuses on building the core infrastructure for large-scale GenAI systems, designing scalable APIs, distributed data systems, and robust deployment pipelines to ensure production-grade reliability and performance. It's an opportunity to solve hard backend and infrastructure challenges that enable AI to work at enterprise scale and shape how AI systems are deployed and scaled in the real world.

Salary not listed

Together AI

At Together AI, you’ll build and operate one of the world’s largest GPU fleets used for frontier model training and inference. This role is for engineers who love building systems, automating everything, and solving problems at massive scale. If you enjoy writing software more than clicking dashboards, obsess over eliminating manual work, and want to build infrastructure that manages tens of thousands of GPUs autonomously, we’d love to talk.

Salary not listed

sarvam

Sarvam is building Indic-language AI for large-scale use, and the CPU Infrastructure team is responsible for all systems outside the GPU fleet. This includes managing Kubernetes across multiple cloud providers, the network infrastructure, identity and access management, observability, cost optimization, and the product delivery pipeline. The team operates on an end-to-end ownership model, where engineers are empowered to automate repetitive manual tasks by building scripts, pipelines, bots, or AI agents. As an AI company, Sarvam integrates AI agents into its own operations for tasks like planning infrastructure changes, triaging alerts, and handling routine issues, with engineers providing oversight. In this role, you will ensure the smooth operation of Sarvam's product infrastructure, handling daily tasks such as deployments, provisioning, access control, incident response, and cost management. A key expectation is to automate these operational duties. The ideal candidate possesses an instinct for identifying manual processes and building automated solutions, whether through scripts, pipeline steps, bots, or AI agents. As you reduce operational toil, your focus will shift towards platform and automation development. You will contribute to the infrastructure supporting a variety of AI products, including voice agents, coding agents, and API platforms.

Salary not listed

Nebius is building a full-stack AI cloud platform for the global AI economy, supporting developers and enterprises from data and model training through to production deployment. We are seeking a Senior Software Engineer to join our Nebius Serverless AI team, focusing on our GPU-native platform for deploying inference endpoints, batch jobs, and AI workloads without infrastructure management. This is a high-ownership role where you will define the technical direction, own the architecture of your contributions, and drive critical design decisions. The work involves end-to-end ownership, from runtime performance to customer-facing APIs, aiming to make our platform the fastest, cheapest, and most reliable for GPU workloads.

Salary not listed

Nebius is building a full-stack AI cloud platform for the global AI economy, supporting developers and enterprises from data and model training through to production deployment. Our Token Factory team is developing an inference platform for various foundation models, aiming to make them fast, reliable, and effortless to deploy at massive scale. We are seeking an engineer to ensure the flawless performance and graceful recovery of this platform under extreme load. This role involves owning the reliability, performance, and observability of the inference stack, from designing telemetry pipelines to tuning Kubernetes autoscalers and crafting resilient infrastructure. You will be instrumental in detecting, isolating, and remediating incidents quickly, and driving a post-mortem culture to prevent recurrence, all while meeting aggressive cost and reliability targets for scaling the platform.

Salary not listed

Nebius is building a full-stack AI cloud platform for the global AI economy, supporting developers and enterprises from data training to production deployment. Tavily, a part of Nebius, is developing the infrastructure layer for agentic web interaction at scale, powering Retrieval-Augmented Generation (RAG) and real-time reasoning in AI systems. This role is for a Senior Site Reliability Engineer who will manage Kubernetes clusters, own infrastructure as code, maintain CI/CD pipelines, and optimize real-time data pipelines. The engineer will also be responsible for building monitoring and alerting systems, debugging production issues, managing cloud costs, and collaborating closely with a small engineering team to own the entire infrastructure.

Salary not listed

Nebius is building a full-stack AI cloud platform to support developers and enterprises from data and model training through to production deployment. We are seeking a Senior HPC Cluster Engineer to join our GPU & InfiniBand team. This role is crucial for enhancing and optimizing the core components of our Cloud platform, with a specific focus on GPU computing, InfiniBand networks, and the KVM/QEMU stack. You will work with hardware virtualization and device emulation technologies to ensure high performance and security in multi-GPU, HPC environments. The position involves analyzing, troubleshooting, and improving infrastructure to support new hardware, fine-tuning system performance, and automating fault detection and resolution in a complex system.

Salary not listed

Together AI

We are seeking a software engineer to develop a Kubernetes-native control plane for managing our GPU inference fleet. This role involves designing a manifest-driven API that allows the inference team to declare their needs for clusters, model deployments, or capacity changes. Our controllers will handle the underlying reconciliation, provider/runtime selection, and lifecycle management, abstracting away the complexities of the serving stack, scheduler, and hardware pool. You will also build systems to optimize fleet efficiency, including defragmentation, rebalancing logic for consolidating workloads, and scheduling/bin-packing improvements to maximize GPU utilization without impacting latency. The primary goal is to decouple platform users from operational complexities while increasing usable capacity from existing hardware. This includes developing controllers, reconciliation loops, and self-service interfaces (API/CLI) for seamless decoupling, alongside event-driven systems for health monitoring, remediation, and utilization management without manual intervention. A product mindset is crucial, with experience building internal platforms or APIs for other engineering teams and a focus on developer experience and ownership of delivered software, including production support.

Salary not listed

Glean is seeking a Software Engineer to join our Platform Infrastructure team, which focuses on customer setup and deployment, particularly in complex cloud, hosting, and security environments. You will help build the systems that bootstrap cloud accounts and projects, wire authentication, and transform deployment workflows into reliable, repeatable productized experiences for customers. This role is ideal for an engineer who enjoys cloud infrastructure, networking, infrastructure as code, and solving real-world deployment problems with clean software abstractions.

$200k - $270k

Together AI

In this role, you will operate, scale, and optimize multi-petabyte storage systems purpose-built for the world’s largest AI training and inference workloads. You’ll manage and scale high-performance parallel filesystems and object stores, evaluate and integrate cutting-edge technologies, and solve complex engineering challenges of operating at extreme throughput, low-latency data paths, and massive cluster-scale storage operations. You will also build Kubernetes-native storage operators and self-service platforms that provide automated provisioning, strict multi-tenancy, performance isolation, and quota enforcement at cluster scale.

$250k - $330k

Glean is seeking a Senior Site Reliability Engineer to enhance the reliability, scalability, and operational excellence of its production services. This role involves designing, building, and operating resilient cloud infrastructure, improving service reliability against defined SLOs, and assisting teams in responding to production issues through effective observability and incident management practices. You will provide technical expertise for system and architectural decisions and collaborate with engineers across the organization to deliver dependable systems. Glean operates in a hybrid cloud environment and faces complex distributed-systems challenges due to its rapid growth. Your work will involve using software engineering and SRE skills to automate tasks, build tools, and improve deployment, monitoring, and on-call workflows, ultimately helping Glean scale and provide a reliable customer experience.

$200k - $260k

Lightning AI is seeking an AI Platform Support Engineer for their APAC Customer Experience team. This role involves supporting Machine Learning (ML) engineers who manage large-scale training and inference workloads on cloud infrastructure, Kubernetes, and GPU platforms in production. You will act as a technical partner, diagnosing failures, enhancing reliability, and guiding customers through complex distributed systems challenges. The issues span Kubernetes scheduling, GPU orchestration, distributed PyTorch failures, inference latency, networking, storage performance, and platform reliability. This position offers exposure to diverse AI workloads and the opportunity to influence the infrastructure powering next-generation ML applications.

Salary not listed

Cohere Health

As a DevOps and Site Reliability Engineer (SRE), you will work closely with application teams to automate and optimize infrastructure, and solve issues in an exciting, fast-paced environment. You will play a vital role in ensuring our systems are reliable, scalable, and high-performing. You will design, implement, and manage robust observability solutions across our cloud infrastructure and applications, comfortable working with metrics, logs, and traces to proactively detect, diagnose, and resolve issues.

Salary not listed

Together AI

As a Technical Support Engineer at a pioneering AI company, you will be the first line of defense supporting customers building training, fine-tuning, and inference solutions. You will address complex technical challenges, provide effective solutions, and act as a product expert. Collaborating with product and sales teams, you will contribute to the continuous improvement of offerings. This role is for a technically proficient individual passionate about AI and customer success, seeking to make a significant impact in a fast-paced, innovative environment.

$160k - $230k

Together AI

We are seeking a software engineer to develop a Kubernetes-native control plane for managing our GPU inference fleet. This role involves designing a manifest-driven API that allows the inference team to declare their needs for clusters, model deployments, or capacity changes. Our controllers will handle the underlying reconciliation, provider/runtime selection, and lifecycle management, abstracting away the complexities of the serving stack, scheduler, and hardware. You will also build systems to optimize fleet efficiency, including defragmentation, rebalancing logic for consolidating workloads, and scheduling/bin-packing improvements to maximize GPU utilization without impacting latency. The primary goal is to decouple users of the platform from operational complexities while maximizing hardware capacity. This includes building controllers, reconciliation loops, and self-service APIs/CLIs, along with event-driven systems for health, remediation, and utilization, all designed to operate without human intervention. A product mindset is crucial, focusing on the developer experience of internal platforms and APIs, with a 'build it, own it' philosophy that includes production operation and support.

Salary not listed

Baseten

Baseten is seeking a Backend/Distributed Systems Engineer to join their Product Platform team. This role is central to Baseten's mission of powering mission-critical AI inference for leading AI companies. You will work on key areas such as commercialization, billing, partnerships, developer productivity, agentic development, and the observability platform. The position offers the opportunity to design and build scalable systems, own core platforms, and contribute to the company's growth at the forefront of AI development.

$165k - $330k