Senior AI Infrastructure Engineer Jobs

Anthropic

Anthropic is seeking a Staff+ Software Engineer to enhance the vector search infrastructure powering Claude. This role is crucial for Claude's ability to access information, impacting web search, conversation memory, and model training. You will join a small, foundational team responsible for the architecture, performance, scalability, and cost of the vector database layer. As a hands-on engineer, you will contribute to coding and make key architectural decisions, shaping the future evolution of this critical system within the Databases organization.

$320k - $485k

NVIDIA is seeking a Senior GPU Networking Architect to join their networking software group. This role is crucial for building and improving GPU communication kernels, bridging GPU computing with networking by ensuring communication primitives are developed in sync with GPU hardware capabilities. The successful candidate will contribute to the software foundation for the world's largest AI systems, working in a diverse and supportive environment that fosters innovation and impact.

Salary not listed

NBCUniversal is building the foundation for responsible, scalable AI across one of the world's most recognized media and entertainment portfolios. As a Senior AI Governance Engineer, you'll help define how AI products are governed, secured, monitored, and operationalized across sports, news, entertainment, film, and digital businesses. You will design and maintain semantic models, partner with data owners to define metrics and lineage, and build CI/CD pipelines and Infrastructure-as-Code solutions supporting AI applications and ML workloads. You will also create monitoring frameworks for various aspects of AI operational health and define standards for auditability, compliance, and responsible AI adoption.

$115k - $145k

Together AI

We are seeking a software engineer to develop a Kubernetes-native control plane for provisioning and managing our GPU inference fleet. This role involves designing a manifest-driven API that allows the inference team to declare their needs, such as clusters, model deployments, or capacity adjustments. Our controllers will handle the underlying reconciliation, provider/runtime selection, and lifecycle management, abstracting away the complexities of the serving stack, scheduler, or hardware pool. You will also build systems to ensure fleet efficiency, including defragmentation, rebalancing logic for consolidating workloads, and scheduling/bin-packing improvements to maximize GPU utilization without compromising latency. The core objective is to decouple platform users from operational complexities while optimizing hardware capacity. This includes developing controllers, reconciliation loops, and self-service interfaces (API/CLI) for seamless operation, alongside event-driven systems for health monitoring, remediation, and utilization management, all designed to operate without human intervention. A product-oriented mindset is crucial, with a focus on building internal platforms and APIs that enhance the developer experience for consuming engineering teams. This role operates under a "you build it, you own it" philosophy, encompassing both software delivery and production support.

$160k - $280k

Cresta

As a member of the infrastructure team, you will be responsible for designing, building, and advancing our core infrastructure. This role supports the engineering team in executing quickly, productively, and securely within a collaborative yet highly autonomous working environment. Each team member has a defined role with clear expectations and the freedom to pursue interesting projects.

$205k - $270k

Typeface

We help the world’s biggest brands move from brief to fully personalized campaigns — in days, not months. Founded by Abhay Parasnis and backed by Microsoft, GV, Salesforce, Lightspeed, Madrona and Menlo, we’re building category-defining technology at the intersection of creativity and AI with real impact. Join us to help shape the future of enterprise marketing. We're looking for a Staff Infrastructure Engineer to help lead core parts of our platform. You'll design, build, and scale the cloud infrastructure that powers our AI and GenAI workloads — ensuring reliability, security, and performance as we grow. This is your chance to shape the backbone of a platform trusted by Fortune 100 companies — building systems that are resilient, observable, and ready to scale with the next wave of AI.

$210k - $240k

As a Principal Software Engineer, you will set the technical direction for NVIDIA's global Developer Compute platform, which encompasses AI agent sandboxes and virtual desktop infrastructure. You will architect and build services to provision, stream, secure, and operate cloud-hosted compute environments at a global scale across AWS and Azure. These environments are designed to support both AI agents and the people who work with them. Your role involves partnering with engineering, IT, and security leaders across NVIDIA to transform AI and end-user workflows into reliable, high-performance platform capabilities, defining how agents and people access compute at the forefront of AI.

$248k - $391k

WPP is seeking a Senior ML Infrastructure Engineer to join their Open Intelligence team in Copenhagen. This hybrid role focuses on building and maintaining scalable, production-grade infrastructure for data and ML workloads, primarily on GCP. You will collaborate with data scientists and engineers to translate research prototypes into robust systems, ensuring reliability, scalability, and operational maturity. The role involves identifying and addressing technical debt, contributing to engineering best practices, and participating in knowledge sharing to drive continuous improvement. This position is crucial for supporting the global expansion of WPP's infrastructure and deepening integrations with AdTech partners, enabling advanced AI work to deliver real-world impact.

Salary not listed

Egen

As a Senior AI Engineer, you will be at the forefront of our Generative AI initiatives, treating AI as a software engineering discipline. You will be responsible for the full lifecycle of our AI features, taking them from initial prototype to robust, scalable production services, while solving for real-world constraints like latency, error handling, and cost optimization. You’ll collaborate with a diverse range of clients to translate business needs into high-performance AI architectures. This role requires a blend of deep technical expertise in LLMs and a disciplined Software Engineering approach to ensure our solutions are robust, ethical, and scalable.

$159k - $219k

The Local AI Inference team is seeking a Senior System Software Engineer to lead the development of robust system-level software and infrastructure for deploying, benchmarking, and qualifying high-quality AI applications and models across Linux and Windows systems powered by RTX and DGX-class GPUs. You will partner with inference engineers, model developers, and AI application teams to build reliable, scalable infrastructure that enables reproducible evaluation and improves local AI performance. Using AI tools and developing autonomous agents, you will accelerate infrastructure development, automate operations, and help engineers diagnose complex system issues.

Salary not listed

NVIDIA is seeking a Senior AI Infrastructure Test Development Engineer to join a mission of crafting distributed technology for managing tens of thousands of resources across NVIDIA teams. This role focuses on enabling engineers to concentrate on development and automation by simplifying operational tasks. The engineer will be responsible for creating test plans, execution, and reporting, alongside developing scripts for test automation, designing QA tools, and creating integration tests. The ideal candidate will leverage AI development tools to enhance quality and productivity throughout the QA workflow, including test automation, code generation, and debugging.

Salary not listed

NVIDIA's Performance Lab (PerfLab) is seeking a System Software Architect to define the technical direction for platforms that evaluate the performance and quality of accelerated computing workloads. This role involves building platform components, leading projects from discovery to deployment, and collaborating with global NVIDIA teams to improve how AI performance is measured and understood. The architect will shape the future of large language models, agentic AI, and emerging HPC and AI workloads.

Salary not listed

d-Matrix

d-Matrix is at the forefront of software and hardware innovation, aiming to unleash the potential of generative AI. We foster a culture of respect, collaboration, humility, and direct communication, valuing diverse perspectives to create better solutions. We are seeking passionate, execution-driven individuals ready to tackle challenges and shape the future of AI. The Inference Cloud Engineering team is responsible for the deployment and orchestration technology behind d-Matrix’s inference services, including an internal cloud for model testing and a customer-facing cloud with self-service inference endpoints. This role will own the end-to-end testing and release process for this critical software, ensuring features are proven, releases are predictable, and rollbacks are safe, even with scarce and shared hardware resources.

$155k - $235k

JPMorgan Chase's Chief Data & Analytics (AIML Data Platforms) team is seeking a Senior Lead Software Engineer to build AI foundation services for GenAI and ML at an enterprise scale. This role involves leading the hands-on delivery of secure, reliable, cloud-native platform capabilities, including Kubernetes, CI/CD, and Infrastructure as Code. You will partner with application teams to create reusable integrations, reference implementations, and onboarding assets, driving significant business impact through deep technical expertise and problem-solving across diverse technologies.

Salary not listed

NVIDIA is seeking a Senior Manager, Software Engineering to lead the evolution of its DevOps, Build Engineering, and Infrastructure team. This role involves building and guiding a high-performing engineering team responsible for the development, testing, verification, and release of core software for NVIDIA's CUDA driver, runtime, and related libraries in C++, Python, and Rust. The position offers hands-on leadership, focusing on people development, CI/CD, technical strategy, architecture, and end-to-end operational ownership, aiming to enhance the developer experience and platform for a global workforce.

$272k - $431k

PALO IT is seeking a DevOps Engineer in CDMX to design, implement, and maintain scalable and efficient cloud infrastructure on AWS. This role involves developing CI/CD pipelines for software deployment, automating tasks with Infrastructure as Code tools, and monitoring application performance. The engineer will collaborate with development teams to integrate new technologies and ensure compliance with security standards, while also maintaining documentation for infrastructure and processes. PALO IT is a global technology consultancy with a trademarked, AI-first approach, Gen-e2™, redefining how enterprises build digital products and transform organizations with AI.

Salary not listed

Cohere

Cohere is seeking a highly experienced Back-End Software Engineer to join their core systems team, focusing on the Model Vault platform. This platform is Cohere's inference solution for deploying and operating AI models securely at enterprise scale. The role involves defining and evolving the architecture of systems that handle identity, deployments, metering, billing, security, privacy, and confidential computing. You will work on scaling these systems to support millions of users and demanding enterprise workloads, ensuring reliability, security, performance, and simplicity in a domain with unique challenges.

$205k - $315k

NVIDIA is developing the foundational software for the next generation of AI infrastructure, focusing on evolving Kubernetes and the cloud-native ecosystem to support advanced AI workloads. The team's mission is to establish Kubernetes as the premier platform for AI, actively contributing upstream to Kubernetes and CNCF projects. As a Senior Software Engineer, you will address complex challenges at the intersection of Kubernetes, distributed systems, and AI infrastructure, working with open-source communities to shape the future of cloud-native AI.

Salary not listed

NVIDIA Cloud Functions is seeking a motivated AI/ML Engineer with expertise in AI platform engineering at scale. The team develops and manages a serverless deployment platform designed to enable and scale AI inferencing workloads. This platform utilizes globally distributed orchestration of workloads on Kubernetes clusters backed by GPUs, operating in a cloud-agnostic manner. The role involves working with a team of innovative engineers to deliver a top-tier product for both external clients and internal NVIDIA teams, contributing to the forefront of cloud engineering paradigms for AI at scale.

$152k - $288k

You.com is building the AI Search Infrastructure that powers modern AI systems, aiming to create a trusted knowledge layer for agents, applications, and enterprises. Our platform combines proprietary vertical indexes with LLM-optimized retrieval systems. We are seeking a Senior Backend Engineer to own and scale the inference gateway, which is the critical layer that handles all model calls. This role involves routing, fallback, caching, metering, and enforcement within strict latency budgets, sitting between model serving and product development.

$200k - $250k

Anthropic

Our Inference team is responsible for building and maintaining the critical systems that serve Claude to millions of users worldwide. We bring Claude to life by serving our models via the industry's largest compute-agnostic inference deployments. We are responsible for the entire stack from intelligent request routing to fleet-wide orchestration across diverse AI accelerators. The team has a dual mandate: maximizing compute efficiency to serve our explosive customer growth, while enabling breakthrough research by giving our scientists the high-performance inference infrastructure they need to develop next-generation models. We tackle complex, distributed systems challenges across multiple accelerator families and emerging AI hardware running in multiple cloud platforms.

€295k - €355k

Anthropic

Our Inference team is responsible for building and maintaining the critical systems that serve Claude to millions of users worldwide. We bring Claude to life by serving our models via the industry's largest compute-agnostic inference deployments. We are responsible for the entire stack from intelligent request routing to fleet-wide orchestration across diverse AI accelerators. The team has a dual mandate: maximizing compute efficiency to serve our explosive customer growth, while enabling breakthrough research by giving our scientists the high-performance inference infrastructure they need to develop next-generation models. We tackle complex, distributed systems challenges across multiple accelerator families and emerging AI hardware running in multiple cloud platforms.

€235k - €295k

NVIDIA is seeking software engineers to work on the CUDA driver for Tegra. CUDA is NVIDIA's platform for accelerating general-purpose computation on the GPU. Our team delivers features and improvements to realize the potential of NVIDIA hardware for a range of computational workloads, from deep learning and scientific computation to video games and virtual reality. CUDA defines a unified programming model across various system configurations and hardware capabilities. The CUDA driver interacts with GPU hardware, kernel mode drivers, user mode drivers, and the operating system. As a member of this team, you will use your design abilities, coding expertise, and creativity to deliver the best compute platform in the world, crafting elegant solutions and shaping the future direction of CUDA.

Salary not listed

The Local AI Automation team is seeking a Senior System Software Engineer passionate about building and maintaining robust system-level software and infrastructure for deploying, benchmarking, and qualifying high-quality AI applications and models. You will collaborate with a team of highly qualified engineers dedicated to creating top-tier infrastructure for Local AI automation. If you have a real passion for building reliable, scalable automation systems in the AI ecosystem, this could be an excellent opportunity.

Salary not listed