AI Infrastructure Engineer Jobs

Together AI

We are seeking a software engineer to develop a Kubernetes-native control plane for provisioning and managing our GPU inference fleet. This role involves designing a manifest-driven API that allows the inference team to declare their needs, such as clusters, model deployments, or capacity adjustments. Our controllers will handle the underlying reconciliation, provider/runtime selection, and lifecycle management, abstracting away the complexities of the serving stack, scheduler, or hardware pool. You will also build systems to ensure fleet efficiency, including defragmentation, rebalancing logic for consolidating workloads, and scheduling/bin-packing improvements to maximize GPU utilization without compromising latency. The core objective is to decouple platform users from operational complexities while optimizing hardware capacity. This includes developing controllers, reconciliation loops, and self-service interfaces (API/CLI) for seamless operation, alongside event-driven systems for health monitoring, remediation, and utilization management, all designed to operate without human intervention. A product-oriented mindset is crucial, with a focus on building internal platforms and APIs that enhance the developer experience for consuming engineering teams. This role operates under a "you build it, you own it" philosophy, encompassing both software delivery and production support.

$160k - $280k

Cresta

As a member of the infrastructure team, you will be responsible for designing, building, and advancing our core infrastructure. This role supports the engineering team in executing quickly, productively, and securely within a collaborative yet highly autonomous working environment. Each team member has a defined role with clear expectations and the freedom to pursue interesting projects.

$205k - $270k

Anthropic

Our Sandboxing team builds the isolation layer that enables Claude to run code, use computers, or act as an agent. These systems are critical for executing code written by Claude for users, training and evaluating models, and supporting customer-deployed agents. The sandboxes must start rapidly, withstand adversarial code, and scale to handle numerous concurrent sessions. This role involves working at the intersection of the Linux kernel, virtualization, and container technologies, collaborating with research, product, and security teams to ensure both safety and performance.

$320k - $485k

Prime Intellect

Prime Intellect is building the open superintelligence stack, providing infrastructure for AI labs. Their platform, Lab, unifies compute, environments, evaluations, and training/deployment for post-training at scale, including SFT, RL, and agent workflows. They are developing open-source models for long-horizon tasks and the platform their research team uses. The company has raised significant funding and is looking for individuals passionate about frontier research, infrastructure, and go-to-market strategies in a nascent category.

$150k - $350k

Typeface

We help the world’s biggest brands move from brief to fully personalized campaigns — in days, not months. Founded by Abhay Parasnis and backed by Microsoft, GV, Salesforce, Lightspeed, Madrona and Menlo, we’re building category-defining technology at the intersection of creativity and AI with real impact. Join us to help shape the future of enterprise marketing. We're looking for a Staff Infrastructure Engineer to help lead core parts of our platform. You'll design, build, and scale the cloud infrastructure that powers our AI and GenAI workloads — ensuring reliability, security, and performance as we grow. This is your chance to shape the backbone of a platform trusted by Fortune 100 companies — building systems that are resilient, observable, and ready to scale with the next wave of AI.

$210k - $240k

As a Principal Software Engineer, you will set the technical direction for NVIDIA's global Developer Compute platform, which encompasses AI agent sandboxes and virtual desktop infrastructure. You will architect and build services to provision, stream, secure, and operate cloud-hosted compute environments at a global scale across AWS and Azure. These environments are designed to support both AI agents and the people who work with them. Your role involves partnering with engineering, IT, and security leaders across NVIDIA to transform AI and end-user workflows into reliable, high-performance platform capabilities, defining how agents and people access compute at the forefront of AI.

$248k - $391k

Egen

As a Senior AI Engineer, you will be at the forefront of our Generative AI initiatives, treating AI as a software engineering discipline. You will be responsible for the full lifecycle of our AI features, taking them from initial prototype to robust, scalable production services, while solving for real-world constraints like latency, error handling, and cost optimization. You’ll collaborate with a diverse range of clients to translate business needs into high-performance AI architectures. This role requires a blend of deep technical expertise in LLMs and a disciplined Software Engineering approach to ensure our solutions are robust, ethical, and scalable.

$159k - $219k

The Local AI Inference team is seeking a Senior System Software Engineer to lead the development of robust system-level software and infrastructure for deploying, benchmarking, and qualifying high-quality AI applications and models across Linux and Windows systems powered by RTX and DGX-class GPUs. You will partner with inference engineers, model developers, and AI application teams to build reliable, scalable infrastructure that enables reproducible evaluation and improves local AI performance. Using AI tools and developing autonomous agents, you will accelerate infrastructure development, automate operations, and help engineers diagnose complex system issues.

Salary not listed

NVIDIA is seeking a Senior AI Infrastructure Test Development Engineer to join a mission of crafting distributed technology for managing tens of thousands of resources across NVIDIA teams. This role focuses on enabling engineers to concentrate on development and automation by simplifying operational tasks. The engineer will be responsible for creating test plans, execution, and reporting, alongside developing scripts for test automation, designing QA tools, and creating integration tests. The ideal candidate will leverage AI development tools to enhance quality and productivity throughout the QA workflow, including test automation, code generation, and debugging.

Salary not listed

NVIDIA's Performance Lab (PerfLab) is seeking a System Software Architect to define the technical direction for platforms that evaluate the performance and quality of accelerated computing workloads. This role involves building platform components, leading projects from discovery to deployment, and collaborating with global NVIDIA teams to improve how AI performance is measured and understood. The architect will shape the future of large language models, agentic AI, and emerging HPC and AI workloads.

Salary not listed

d-Matrix

d-Matrix is at the forefront of software and hardware innovation, aiming to unleash the potential of generative AI. We foster a culture of respect, collaboration, humility, and direct communication, valuing diverse perspectives to create better solutions. We are seeking passionate, execution-driven individuals ready to tackle challenges and shape the future of AI. The Inference Cloud Engineering team is responsible for the deployment and orchestration technology behind d-Matrix’s inference services, including an internal cloud for model testing and a customer-facing cloud with self-service inference endpoints. This role will own the end-to-end testing and release process for this critical software, ensuring features are proven, releases are predictable, and rollbacks are safe, even with scarce and shared hardware resources.

$155k - $235k

NVIDIA is seeking a Senior Manager, Software Engineering to lead the evolution of its DevOps, Build Engineering, and Infrastructure team. This role involves building and guiding a high-performing engineering team responsible for the development, testing, verification, and release of core software for NVIDIA's CUDA driver, runtime, and related libraries in C++, Python, and Rust. The position offers hands-on leadership, focusing on people development, CI/CD, technical strategy, architecture, and end-to-end operational ownership, aiming to enhance the developer experience and platform for a global workforce.

$272k - $431k

Cohere

Cohere is seeking a highly experienced Back-End Software Engineer to join their core systems team, focusing on the Model Vault platform. This platform is Cohere's inference solution for deploying and operating AI models securely at enterprise scale. The role involves defining and evolving the architecture of systems that handle identity, deployments, metering, billing, security, privacy, and confidential computing. You will work on scaling these systems to support millions of users and demanding enterprise workloads, ensuring reliability, security, performance, and simplicity in a domain with unique challenges.

$205k - $315k

NVIDIA is developing the foundational software for the next generation of AI infrastructure, focusing on evolving Kubernetes and the cloud-native ecosystem to support advanced AI workloads. The team's mission is to establish Kubernetes as the premier platform for AI, actively contributing upstream to Kubernetes and CNCF projects. As a Senior Software Engineer, you will address complex challenges at the intersection of Kubernetes, distributed systems, and AI infrastructure, working with open-source communities to shape the future of cloud-native AI.

Salary not listed

NVIDIA Cloud Functions is seeking a motivated AI/ML Engineer with expertise in AI platform engineering at scale. The team develops and manages a serverless deployment platform designed to enable and scale AI inferencing workloads. This platform utilizes globally distributed orchestration of workloads on Kubernetes clusters backed by GPUs, operating in a cloud-agnostic manner. The role involves working with a team of innovative engineers to deliver a top-tier product for both external clients and internal NVIDIA teams, contributing to the forefront of cloud engineering paradigms for AI at scale.

$152k - $288k

Reflection ai

Reflection is a research lab dedicated to making intelligence open and accessible. We build open models that empower users to control their intelligence and shape the future of AI. In this role, you will develop the critical evaluation infrastructure that researchers rely on to understand model capabilities, inform training decisions, and assess readiness for release. Your work will encompass evaluation authoring, distributed execution, and results analysis, with a specific emphasis on pre-training. You will collaborate closely with researchers to enhance the speed, reliability, and ease of use of evaluations, ensuring that the outcomes accurately reflect the models being assessed.

Salary not listed

You.com is building the AI Search Infrastructure that powers modern AI systems, aiming to create a trusted knowledge layer for agents, applications, and enterprises. Our platform combines proprietary vertical indexes with LLM-optimized retrieval systems. We are seeking a Senior Backend Engineer to own and scale the inference gateway, which is the critical layer that handles all model calls. This role involves routing, fallback, caching, metering, and enforcement within strict latency budgets, sitting between model serving and product development.

$200k - $250k

Anthropic

Our Inference team is responsible for building and maintaining the critical systems that serve Claude to millions of users worldwide. We bring Claude to life by serving our models via the industry's largest compute-agnostic inference deployments. We are responsible for the entire stack from intelligent request routing to fleet-wide orchestration across diverse AI accelerators. The team has a dual mandate: maximizing compute efficiency to serve our explosive customer growth, while enabling breakthrough research by giving our scientists the high-performance inference infrastructure they need to develop next-generation models. We tackle complex, distributed systems challenges across multiple accelerator families and emerging AI hardware running in multiple cloud platforms.

€295k - €355k

sarvam

Sarvam is building India's full-stack sovereign AI platform, focusing on research, models, infrastructure, and applications to make AI work for India. We partner with leading enterprises and public institutions, backed by top venture capital firms. We are seeking a Software Engineer to own major parts of our model APIs platform end-to-end. This platform handles millions of daily API calls for services like ASR, TTS, Vision, LLM, and Translate, including features like metering, billing, and rate-limiting. You will be responsible for the reliability, performance, and low latency of critical components, working closely with our Staff Engineer.

Salary not listed

Anthropic

Our Inference team is responsible for building and maintaining the critical systems that serve Claude to millions of users worldwide. We bring Claude to life by serving our models via the industry's largest compute-agnostic inference deployments. We are responsible for the entire stack from intelligent request routing to fleet-wide orchestration across diverse AI accelerators. The team has a dual mandate: maximizing compute efficiency to serve our explosive customer growth, while enabling breakthrough research by giving our scientists the high-performance inference infrastructure they need to develop next-generation models. We tackle complex, distributed systems challenges across multiple accelerator families and emerging AI hardware running in multiple cloud platforms.

€235k - €295k

NVIDIA is seeking software engineers to work on the CUDA driver for Tegra. CUDA is NVIDIA's platform for accelerating general-purpose computation on the GPU. Our team delivers features and improvements to realize the potential of NVIDIA hardware for a range of computational workloads, from deep learning and scientific computation to video games and virtual reality. CUDA defines a unified programming model across various system configurations and hardware capabilities. The CUDA driver interacts with GPU hardware, kernel mode drivers, user mode drivers, and the operating system. As a member of this team, you will use your design abilities, coding expertise, and creativity to deliver the best compute platform in the world, crafting elegant solutions and shaping the future direction of CUDA.

Salary not listed

The Local AI Automation team is seeking a Senior System Software Engineer passionate about building and maintaining robust system-level software and infrastructure for deploying, benchmarking, and qualifying high-quality AI applications and models. You will collaborate with a team of highly qualified engineers dedicated to creating top-tier infrastructure for Local AI automation. If you have a real passion for building reliable, scalable automation systems in the AI ecosystem, this could be an excellent opportunity.

Salary not listed

Physical Intelligence

Physical Intelligence is building general-purpose AI for the physical world, developing foundation models and learning algorithms for robots and future physically-actuated devices. The Infrastructure team is central to this mission, building and operating the backbone for training state-of-the-art VLA models, large-scale simulation, and reliable deployment of intelligence across robot fleets. This role involves collaborating closely with researchers, robotics runtime, product, and platform engineers to ensure infrastructure scales from prototype to production-grade deployments.

Salary not listed

Code Metal

Code Metal is seeking a Senior Software Engineer for its AI Platform team. This role involves designing, building, and operating the foundational AI enablement stack that powers the company's AI-driven code solutions. You will contribute to critical components such as GPU inference serving, model gateways, agent harnesses, context engineering, observability, and AI experimentation management. While this is an internal platform, it is built to product standards. The position offers the opportunity to eventually own a specific area, like model serving or agentic infrastructure, while also contributing broadly across the stack. The role requires a blend of strong software engineering skills with solid data science and AI research fundamentals, including occasional experimental work to inform platform decisions.

$170k - $210k