AI Infrastructure Engineer Jobs

Modal is building a new infrastructure layer for AI, focusing on the demands of AI workloads that previous infrastructure couldn't support. Our platform provides instant GPU access, sub-second container starts, and native storage, enabling customers to serve low-latency inference, fine-tune models, and access production-ready sandboxes at scale. We are experiencing rapid growth, with a significant increase in annualized run rate and a recent Series D funding round. Our team comprises experienced engineers and researchers from diverse backgrounds, including creators of popular open-source projects and academic researchers.

$220k - $300k

Wayve's AI Enablement team builds the shared platforms that enable employees across the company to use language models and AI agents safely, reliably, and cost-effectively. As an early member of this platform team, you will help shape both the technical foundations and the standards for how AI-powered tools are built and operated across Wayve. You will design, build, and operate infrastructure for model access, agent runtimes, and observability. You will take projects from initial problem definition through technical design, deployment, and ongoing operation, partnering closely with security, IT, and engineering teams. You will also support teams adopting the platform by troubleshooting issues, documenting reusable patterns, and helping people understand what's possible with the tools available.

Salary not listed

Cerebras

Cerebras Systems is seeking a senior software engineer to build and scale the platforms and tools used to monitor and manage its growing fleet of AI clusters across multiple data centers. This role involves owning substantial engineering work across backend services, operational tools, and user-facing applications. You will collaborate with various teams, including Cluster Operations, infrastructure, inference, security, and data center teams, to translate operational challenges into reliable, scalable software solutions. The goal is to ensure fleet health, efficient issue response, and continuous compute capacity availability.

Salary not listed

Together AI

Together AI is seeking a Staff Platform Infrastructure Engineer to lead service and platform infrastructure strategy across the engineering organization. This is a hands-on technical leadership role with the opportunity to establish a new team and define the standard for how services are built and operated. You will provide a stable, scalable service infrastructure foundation for mission-critical product and platform teams, focusing on reliability, repeatability, and ease of adoption. You will own and evolve crucial shared infrastructure, roll out new patterns and platform capabilities, and ensure services are built on company-wide foundations.

$240k - $280k

NVIDIA DGX Cloud provides AI services and endpoints for research and production workloads. We are seeking a Senior Production Engineer to develop software and automation that ensures these services are reliable, scalable, and safe to operate. The Production Engineering team focuses on large-scale distributed systems, including internal and external model endpoints, regional control plane services for workload orchestration and request routing, and the GPU/CPU compute infrastructure for inference and agentic workloads. Our work encompasses Kubernetes clusters across various cloud environments (AWS, Azure, Google Cloud, partner clouds) and on-premises deployments.

$184k - $357k

Faire is a technology wholesale platform connecting independent retailers globally. We are seeking a Staff Engineer to own and evolve our data plumbing, ensuring data is accessible, accurate, and queryable for various teams including Product Engineering, Data Science, and Machine Learning. This hands-on role involves designing and building the next generation of our data ingestion and processing systems, and serving as the go-to expert for data movement at Faire.

$247k - $339k

Join us as we build the next generation of enterprise search infrastructure powering AI agents. Our mission is to evolve from traditional product-specific search systems into a composable, agent-native platform that enables agents and applications to discover, retrieve, reason over, and recommend information across heterogeneous enterprise data sources. As a Senior Staff Engineer, you will be the technical owner for foundational search infrastructure that serves as the backbone of our agentic platform. You will drive the architecture, scalability, and evolution of core indexing and retrieval systems while partnering closely with Search Ranking, ML, Platform, and AI teams to deliver production-grade search capabilities that power agent experiences across the company.

$191k - $334k

SambaNova

SambaNova is seeking Principal Software Engineers to join the SambaStack team, focusing on a cutting-edge AI inference serving platform. This platform enables enterprises and service providers to host their own AI inference solutions, leveraging SambaNova's unique RDU architecture. The role involves enhancing the performance, scalability, and reliability of this cloud-agnostic, enterprise-grade platform, which is designed for deploying and managing foundation model workloads at scale. You will be instrumental in introducing new features, building critical components, and driving technical execution for key infrastructure initiatives at the intersection of AI infrastructure, enterprise software, and developer experience.

INR 8900k - INR 10800k

We are expanding our research and development for deep learning and are seeking excellent Software Engineers to join our team. We specialize in developing GPU-accelerated deep learning software, enabling breakthroughs in numerous areas. Join the team that builds software to enable new solutions. Your ability to work in a fast-paced, customer-oriented team is required, and excellent communication skills are necessary.

Salary not listed

d-Matrix

d-Matrix is seeking a Sr Staff Site Reliability Engineer to join their AI Infrastructure team. This role is crucial for owning the infrastructure layer that powers all engineering teams and customers, encompassing colocation facilities, on-premises GPU clusters, cloud environments, and platform services. You will be responsible for reliability, automation, and observability across these diverse environments, managing systems end-to-end from provisioning to incident response. This is a hands-on position focused on building and operating real infrastructure, collaborating with hardware and software teams on CI/CD, QA, and HPC workloads, and supporting customer-facing deployments.

$175k - $265k

NVIDIA is seeking a Senior System Software Architect specializing in AI and GPU Networking to join their AI Networking Research team. This role involves exploring technological challenges in accelerated networking and building AI data centers, researching new transport functions and semantics for optimizing AI workloads and systems, and leading architectural and development efforts across various fields including distributed AI, deep learning, data analytics, High Performance Computing (HPC), Software Defined Networking (SDN), virtualization, and storage. The architect will enhance NVIDIA's GPU Networking offerings, co-design hardware features to accelerate data movement for inference and model serving, identify and evaluate new technologies, and lead the architecture and design of new innovations like runtime systems and communication libraries.

Salary not listed

Anthropic

Anthropic is building reliable, interpretable, and steerable AI systems to be safe and beneficial for users and society. The Cloud Inference team is responsible for scaling and optimizing Claude to serve massive audiences across AWS, GCP, Azure, and other cloud providers. This involves owning the end-to-end product on each platform, from API integration and intelligent request routing to inference execution, capacity management, and operations. Engineers on this team are highly leveraged, driving multiple revenue streams while optimizing compute resources. The role requires product-minded backend engineers who can navigate platform differences, design cross-provider services and abstractions, and make architectural decisions for reliability and cost-effectiveness at massive scale. The work directly contributes to increasing service scale, accelerating the launch of new models and features, and ensuring LLMs meet rigorous safety, performance, and security standards.

$320k - $485k

Anthropic

The Cloud Inference team is responsible for scaling and optimizing Claude to serve massive audiences of developers and enterprise companies across major cloud platforms like AWS, GCP, and Azure. Within this team, the model & inference launch team focuses on the validation pipeline for inference servers and load balancers. This critical infrastructure work ensures that every inference change, including new model launches and performance improvements, is deployed with correctness, performance, and reliability. The goal is to make validation fast, cost-effective, and trustworthy, directly impacting how quickly advanced models and features reach users on all cloud platforms and how efficiently compute resources are utilized.

$320k - $485k

HeyGen is building AI applications including Avatar IV, Photo Avatar, Interactive Avatar, and Video Translation. We are looking for a Software Engineer focused on GPU performance to make the systems behind these experiences faster and more efficient. You will work across model execution and inference infrastructure, using profiling and measurement to improve latency, throughput, and GPU cost. This role is a fit for an engineer who enjoys understanding how software uses the hardware beneath it.

Salary not listed

NVIDIA is seeking AI development interns to join our AI Infra team. This team focuses on enabling NVIDIA and its customers to scale deep learning workflows more efficiently. The work involves building infrastructure for data storage, processing, retrieval, and mining, as well as developing tools for workflow orchestration, issue triage, and model KPI evaluation and testing. The team also handles AI data at a massive scale for applications like autonomous driving.

Salary not listed

Salient

Salient is seeking a Telephony Infrastructure Engineer to take ownership of the voice platform that powers our AI agents. This role involves managing the SIP edge, Asterisk, voice audio and dialing layers, and call servers. You will collaborate with AI agent engineers, telephony specialists, customer success teams, and carriers to ensure the platform's reliability, safety, and scalability for future growth. This is a hands-on position where you will complete ongoing projects, enhance platform stability, and prepare it to handle significantly increased call volumes. Additionally, you will design and implement customer telephony integrations, working towards making these processes more self-serve.

$200k - $300k

Fin, now part of Salesforce, is seeking an Engineering Manager for its AI Models Infrastructure Team. This role is responsible for leading a team of expert engineers who build and operate the foundational infrastructure that empowers the company to train and run its AI models. The ideal candidate will have prior experience in AI and a strong appetite for continuous technical learning, with the ability to manage and grow a high-performing team in a fast-evolving technical domain. This is a highly technical management position focused on enabling AI capabilities and driving innovation.

Salary not listed

Waabi is a leader in Physical AI, developing technology for autonomous transportation. We are seeking a Senior/Staff ML Ops Engineer to build and evolve our training infrastructure on Kubernetes, focusing on GPU scheduling, autoscaling, and multi-node distributed jobs. You will shape the developer-facing surface of our tools, shorten the development loop for ML engineers, and evangelize best-in-class tooling. This role involves strengthening our data and artifact layer, turning one-off scripts into durable, tested libraries, and making experiments legible through robust experiment tracking and model registries. You will also ship CI/CD for models, build comprehensive observability across the ML stack, and treat documentation and onboarding as a core product surface. Driving adoption through user empathy and prototyping is key, alongside ensuring platform reliability and building secure guardrails.

Salary not listed

Together AI

Together AI is building the AI Native Cloud, an end-to-end platform for the full generative AI lifecycle, combining the fastest LLM inference engine with state-of-the-art GPU cloud infrastructure. The Together Cloud team builds the flagship IaaS product that provides high-performance, AI-ready GPU clusters through a self-serve cloud console, along with the virtualized infrastructure layer powering Together's inference, RL, and fine-tuning products. As a Staff Software Engineer focusing on AI Compute, you will set technical direction for and build major components of the next generation AI cloud platform – a highly available, global cloud infrastructure with cutting-edge virtualization of the latest ML hardware. This is an architect-and-build role where you will set the architecture for automated bootstrapping of GPU data centers, high-performance virtualization, and fault-tolerant decentralized control planes, and be a key owner of the hardest parts, in the code as well as the design. Your designs will span the IaaS layer up to the global management plane that schedules capacity across all data centers. At this level, the job is as much leverage as code: the standards you set and the engineers you grow decide how fast the rest of Together Cloud ships.

$260k - $300k

Together AI

Together AI is building the AI Native Cloud, an end-to-end platform for the full generative AI lifecycle, combining a fast LLM inference engine with state-of-the-art GPU cloud infrastructure. The Together Cloud team builds the flagship IaaS product that provides high-performance, AI-ready GPU clusters through a self-serve cloud console, along with the virtualized infrastructure layer powering Together's inference, RL, and fine-tuning products. As a Senior Software Engineer focusing on AI Compute, you will build and own major components of the next generation AI cloud platform – a highly available, global cloud infrastructure with cutting-edge virtualization of the latest ML hardware. This virtualized computing platform powers our own SaaS products and serves external cloud customers through self-serve offerings. The hard problems in rapidly scaling heterogeneous GPU fleets are software problems that we solve by building global and in-DC services, Kubernetes operators, and high-performance SDN libraries, forking hypervisors and Linux kernels, and building infra tailored to inference and fine-tuning. You'll own massive greenfield projects across their full lifecycle – scoping the problem, designing the system, and building it through to GA launch.

$220k - $270k

Together AI

Together AI is building the AI Acceleration Cloud, an end-to-end platform for the full generative AI lifecycle. The AI Infrastructure team is at the forefront of building and scaling the foundational systems that power our generative AI platform. The storage and observability team is crucial for designing, implementing, and maintaining robust distributed storage solutions, ensuring seamless data access and management. They are also responsible for developing comprehensive observability platforms, providing critical insights into system performance and GPU utilization, and proactively identifying and resolving issues.

$200k - $280k

Together AI

Together AI is building the AI Acceleration Cloud, an end-to-end platform for the full generative AI lifecycle. As a Senior Backend Engineer, you will play a key role in building the next generation AI cloud platform – a highly available, global, blazing-fast cloud infrastructure that virtualizes cutting-edge ML hardware and enables state-of-the-art ML practitioners with self-serve AI cloud services. This platform serves both internal StaaS products and external cloud customers across dozens of data centers worldwide.

$160k - $230k

Together AI

Together AI is building the Inference Platform to deliver advanced generative AI models, powering multi-tenant serverless workloads and dedicated endpoints. This platform enables developers, enterprises, and researchers to utilize the latest LLMs and multimodal models at scale. The role focuses on optimizing latency and maximizing the utilization of tens of thousands of GPUs. You will collaborate directly with research teams to productionize frontier models and engage with the open-source community, contributing to and leveraging projects like SGLang and vLLM to enhance inference performance and efficiency.

$200k - $290k

Together AI

As a Systems Research Engineer specialized in GPU Programming, you will play a crucial role in developing and optimizing GPU-accelerated kernels and algorithms for ML/AI applications. You will co-design GPU kernels and model architecture to enhance the performance and efficiency of our AI systems, and contribute to the co-design of efficient GPU architectures and programming models. Your research skills will be vital in staying up-to-date with the latest advancements in GPU programming techniques, ensuring that our AI infrastructure remains at the forefront of innovation.

$200k - $300k