Remote AI Infrastructure Engineer Jobs

We are seeking an enthusiastic software engineer to join our AI networking acceleration team. You will work on a groundbreaking open-source library, utilizing hardware offloads, GPU Kernels, and RDMA network cards. Our product is a performance-oriented, low-level infrastructure designed to revolutionize inference. We foster a collaborative and innovative team environment, offering rewarding opportunities to work with industry leaders on transformative products.

$293k - $507k

Harvey is transforming legal and professional services by combining agentic AI, an enterprise-grade platform, and deep domain expertise. This is an opportunity to join a fast-scaling company at an inflection point, defining a new category in real-time. The team operates with intensity, ownership, and a commitment to excellence, driven by values of decisiveness, simplicity, and a "Job's Not Finished" mentality. As a Member of Technical Staff, Infrastructure in Harvey Research, you will build the systems that power the adaptation and improvement of leading open-weight models for legal work. You will own the infrastructure for scaling post-training efforts, including distributed training, RL rollout systems, agent environments, and internal tools for data management, evaluation, and job tracking. You will collaborate closely with researchers and contribute to decisions about system architecture and adoption.

$230k - $340k

Vercel

As a member of the AI Gateway Team, you will be responsible for developing and enhancing our AI Gateway platform, which provides a unified API for accessing hundreds of AI models from multiple providers. You will collaborate with a remote and distributed team of engineers to build reliable, low-latency systems that handle rate limiting, intelligent failovers, and seamless integrations, enabling developers to ship AI-powered applications without managing provider-specific complexities. This role focuses on ensuring production-ready reliability for AI workloads, including automatic fallbacks during outages and consistent performance across providers like OpenAI, Anthropic, Google, and more.

$196k - $294k

Fal

fal is building the generative media ecosystem for the next generation of AI products, providing the infrastructure, tools, and model access needed to move from idea to production at scale. We offer a unified platform for high-performance inference, orchestration, and observability, enabling new categories of AI-native products. As generative media reshapes industries, fal is becoming the foundation for ambitious teams. This role focuses on empowering fal's ML team by developing automation, infrastructure, and developer tooling to streamline the development, testing, and deployment of generative AI models. You will own and enhance CI/CD systems for a growing collection of ML models and inference pipelines, aiming to eliminate manual work, accelerate development cycles, and build reliable systems for confident model shipping and optimization.

$170k - $230k

NVIDIA is seeking an engineering lead to architect and scale the data acquisition infrastructure that powers our foundation models. This hands-on technical leadership role spans distributed systems, crawl infrastructure, and third-party data ingestion, focusing on building systems that discover valuable sources, crawl the open web at scale, and deliver reliable, high-quality data to training pipelines. You will own the architecture of a unified platform covering source discovery, URL indexing, crawl frontiers, predictive ranking, scheduling, distributed fetching, and ingestion, with the goal of bringing existing infrastructure to pretraining scale and extending it to additional model families, modalities, and acquisition workflows. You will also lead architecture reviews, critical implementation work, mentor engineers, and help build the team.

$272k - $431k

NVIDIA is seeking a Deep Learning Software Infrastructure Engineer to advance its Autonomous Vehicles project. In this role, you will be responsible for building and scaling the training libraries and infrastructure necessary for end-to-end autonomous driving models. By enabling training on thousands of GPUs and massive datasets, you will significantly accelerate iteration speed and enhance safety, working collaboratively with research and platform teams across NVIDIA.

$224k - $431k

AI Infrastructure Engineers at NVIDIA build the systems, tooling, and data infrastructure that enable operation of our GPU cloud services. We are enabling engineering teams to innovate while proactively identifying, tracking, and mitigating risks across the entire technical task. This role is ideal for engineers who thrive at the intersection of product, infrastructure, and software engineering and who want to build automated, intelligence-driven systems that protect NVIDIA’s most critical AI platforms.

$184k - $357k

Turing

Turing is seeking a Senior DevOps & Infrastructure Engineer to join their global Infrastructure Engineering team. This role focuses on building and maintaining the cloud infrastructure and developer platform that powers the company's operations. You will be responsible for making systems more reliable, secure, scalable, and efficient as the company grows. This is a hands-on position for an engineer who thrives on automating manual tasks, solving complex production issues, and elevating operational excellence. You will collaborate closely with application and security teams, working across cloud infrastructure, reliability, developer tooling, and security. The role offers meaningful ownership within a globally distributed team, requiring regular overlap with U.S. Pacific Time working hours and participation in a global on-call rotation.

Salary not listed

Vectara

Vectara is seeking a Senior/Staff Platform Software Engineer to build and scale the core infrastructure powering their AI agent platform. This role involves designing robust services and scalable systems with a focus on automation, observability, reliability, and performance. You will work on AI-assisted coding, lead code and system design reviews, and mentor team members. The position requires designing, implementing, and maintaining scalable and secure backend services, driving infrastructure automation, CI/CD, and monitoring pipelines, and collaborating with product and research teams to deploy and optimize ML/NLP workloads. You will also lead initiatives to improve system reliability and scalability, contribute to architecture decisions, and champion best practices in code quality and DevOps.

Salary not listed

Cohere

Cohere is seeking a Staff Forward Deployed Engineer to bridge the gap between frontier AI capabilities and complex customer realities. This role involves architecting high-performance, resilient machine learning systems within enterprise environments, acting as the primary technical liaison between Cohere's core engineering teams and strategic partners. You will own the architecture for customer-specific inference challenges, ensuring our large language models operate reliably, securely, and at peak performance within client infrastructure. The goal is to achieve exceptionally low latency, high throughput, and seamless resilience in production deployments.

$240k - $415k

Assort Health

Assort Health is building a continuous conversation for every patient, powered by an AI agent that remembers context and personal preferences. This AI-driven healthcare platform aims to revolutionize patient interactions, moving beyond the current system's forgetfulness. The company has experienced significant growth, with substantial funding and recognition, and is seeking individuals passionate about making healthcare more accessible.

$190k - $240k

Nebius is building a full-stack AI cloud platform to support developers and enterprises from data and model training through to production deployment. We are seeking a GPU Cluster Architect to drive the design of our next-generation AI infrastructure. In this high-impact, hands-on role, you will make end-to-end architectural decisions across compute, networking, and storage, ensuring our platforms can meet the massive scale, performance, and reliability requirements of modern AI workloads. You will define how tens of thousands of GPUs are interconnected, cooled down, powered, and optimized across multiple data center sites.

$184k - $318k

Vectara

Vectara is seeking a Platform Engineer to own the infrastructure that runs our deploy-anywhere platform. This role involves managing Kubernetes clusters for ML inference at scale, CI/CD pipelines, infrastructure as code, and the observability stack. You will be hands-on, writing Helm charts and Terraform, debugging issues, and shipping backend service features. The position requires deploying across AWS, GCP, and on-premises environments, including air-gapped setups, and participating in an on-call rotation to support enterprise customers.

Salary not listed

Cresta

As a member of the infrastructure team, you will be responsible for designing, building, and advancing our core infrastructure. This role supports the engineering team in executing quickly, productively, and securely within a collaborative yet highly autonomous working environment. Each team member has a defined role with clear expectations and the freedom to pursue interesting projects.

$205k - $270k

As a Principal Software Engineer, you will set the technical direction for NVIDIA's global Developer Compute platform, which encompasses AI agent sandboxes and virtual desktop infrastructure. You will architect and build services to provision, stream, secure, and operate cloud-hosted compute environments at a global scale across AWS and Azure. These environments are designed to support both AI agents and the people who work with them. Your role involves partnering with engineering, IT, and security leaders across NVIDIA to transform AI and end-user workflows into reliable, high-performance platform capabilities, defining how agents and people access compute at the forefront of AI.

$248k - $391k

Egen

As a Senior AI Engineer, you will be at the forefront of our Generative AI initiatives, treating AI as a software engineering discipline. You will be responsible for the full lifecycle of our AI features, taking them from initial prototype to robust, scalable production services, while solving for real-world constraints like latency, error handling, and cost optimization. You’ll collaborate with a diverse range of clients to translate business needs into high-performance AI architectures. This role requires a blend of deep technical expertise in LLMs and a disciplined Software Engineering approach to ensure our solutions are robust, ethical, and scalable.

$159k - $219k

d-Matrix

d-Matrix is at the forefront of software and hardware innovation, aiming to unleash the potential of generative AI. We foster a culture of respect, collaboration, humility, and direct communication, valuing diverse perspectives to create better solutions. We are seeking passionate, execution-driven individuals ready to tackle challenges and shape the future of AI. The Inference Cloud Engineering team is responsible for the deployment and orchestration technology behind d-Matrix’s inference services, including an internal cloud for model testing and a customer-facing cloud with self-service inference endpoints. This role will own the end-to-end testing and release process for this critical software, ensuring features are proven, releases are predictable, and rollbacks are safe, even with scarce and shared hardware resources.

$155k - $235k

Cohere

Cohere is seeking a highly experienced Back-End Software Engineer to join their core systems team, focusing on the Model Vault platform. This platform is Cohere's inference solution for deploying and operating AI models securely at enterprise scale. The role involves defining and evolving the architecture of systems that handle identity, deployments, metering, billing, security, privacy, and confidential computing. You will work on scaling these systems to support millions of users and demanding enterprise workloads, ensuring reliability, security, performance, and simplicity in a domain with unique challenges.

$205k - $315k

You.com is building the AI Search Infrastructure that powers modern AI systems, aiming to create a trusted knowledge layer for agents, applications, and enterprises. Our platform combines proprietary vertical indexes with LLM-optimized retrieval systems. We are seeking a Senior Backend Engineer to own and scale the inference gateway, which is the critical layer that handles all model calls. This role involves routing, fallback, caching, metering, and enforcement within strict latency budgets, sitting between model serving and product development.

$200k - $250k

Code Metal

Code Metal is seeking a Senior Software Engineer for its AI Platform team. This role involves designing, building, and operating the foundational AI enablement stack that powers the company's AI-driven code solutions. You will contribute to critical components such as GPU inference serving, model gateways, agent harnesses, context engineering, observability, and AI experimentation management. While this is an internal platform, it is built to product standards. The position offers the opportunity to eventually own a specific area, like model serving or agentic infrastructure, while also contributing broadly across the stack. The role requires a blend of strong software engineering skills with solid data science and AI research fundamentals, including occasional experimental work to inform platform decisions.

$170k - $210k

We are seeking a Senior Technical Support Engineer with deep expertise in Ethernet networking and AI infrastructure. In this customer-facing role, you will own complex technical investigations and critical escalations involving large-scale data center and AI environments. You will serve as a trusted technical advisor to strategic customers, combining strong troubleshooting skills with excellent communication, crisis management, and technical leadership. You will collaborate closely with Engineering, Product Management, Marketing, and Support teams to resolve issues and improve our products and support practices.

Salary not listed

Luma is seeking a Software Engineer to own the serving of their AI models. This role involves integrating new model architectures into the inference engine, scaling deployments across thousands of machines, and optimizing the utilization of expensive GPU fleets while meeting internal service level objectives (SLOs). The work focuses on large-scale inference systems, including scheduling, fleet management, deployment pipelines, and ensuring reliability across diverse clusters and hardware providers. This position is ideal for a strong systems engineer comfortable with model serving and Kubernetes at scale, rather than those seeking pure modeling work.

Salary not listed

Luma is seeking a Research Scientist / Engineer to build and optimize the distributed systems that train large-scale multimodal models across thousands of GPUs. This role focuses on advanced parallelism, training stability, and utilization across massive clusters, requiring deep expertise in distributed PyTorch and CUDA. The goal is to provide researchers with reliable, efficient, and scalable infrastructure for innovation.

Salary not listed

Anthropic

Anthropic's Environments organization is responsible for building and maintaining the infrastructure that enhances Claude's capabilities through reinforcement learning. This includes developing frameworks for researchers to build environments and the underlying infrastructure for running them, with a core mission to productionize research. The role involves embedding with research teams to understand their workflows and designing frameworks and APIs that accelerate their progress, ensuring these systems are understandable, ownable, and maintainable. A significant part of the role also includes ensuring the health, maintainability, monitoring, and ease of triage for production RL runs. The ideal candidate will possess deep Python expertise, a strong sense of API and framework design, and an intuitive understanding of complex system failures, particularly silent ones. Experience with stateful distributed systems like workflow engines or actor frameworks is a plus. The role requires comfort in navigating and abstracting complex research codebases, incrementally improving them while research continues, and leveraging AI tools for development with a focus on verification.

$405k - $605k