Remote AI Infrastructure Engineer Jobs

d-Matrix

d-Matrix is at the forefront of software and hardware innovation, aiming to unleash the potential of generative AI. We foster a culture of respect, collaboration, humility, and direct communication, valuing diverse perspectives to create better solutions. We are seeking passionate, execution-driven individuals ready to tackle challenges and shape the future of AI. The Inference Cloud Engineering team is responsible for the deployment and orchestration technology behind d-Matrix’s inference services, including an internal cloud for model testing and a customer-facing cloud with self-service inference endpoints. This role will own the end-to-end testing and release process for this critical software, ensuring features are proven, releases are predictable, and rollbacks are safe, even with scarce and shared hardware resources.

$155k - $235k

Cohere

Cohere is seeking a highly experienced Back-End Software Engineer to join their core systems team, focusing on the Model Vault platform. This platform is Cohere's inference solution for deploying and operating AI models securely at enterprise scale. The role involves defining and evolving the architecture of systems that handle identity, deployments, metering, billing, security, privacy, and confidential computing. You will work on scaling these systems to support millions of users and demanding enterprise workloads, ensuring reliability, security, performance, and simplicity in a domain with unique challenges.

$205k - $315k

You.com is building the AI Search Infrastructure that powers modern AI systems, aiming to create a trusted knowledge layer for agents, applications, and enterprises. Our platform combines proprietary vertical indexes with LLM-optimized retrieval systems. We are seeking a Senior Backend Engineer to own and scale the inference gateway, which is the critical layer that handles all model calls. This role involves routing, fallback, caching, metering, and enforcement within strict latency budgets, sitting between model serving and product development.

$200k - $250k

Code Metal

Code Metal is seeking a Senior Software Engineer for its AI Platform team. This role involves designing, building, and operating the foundational AI enablement stack that powers the company's AI-driven code solutions. You will contribute to critical components such as GPU inference serving, model gateways, agent harnesses, context engineering, observability, and AI experimentation management. While this is an internal platform, it is built to product standards. The position offers the opportunity to eventually own a specific area, like model serving or agentic infrastructure, while also contributing broadly across the stack. The role requires a blend of strong software engineering skills with solid data science and AI research fundamentals, including occasional experimental work to inform platform decisions.

$170k - $210k

FAR.AI is seeking a Tech Lead Manager for its GPU Cluster Infrastructure team. This role involves leading a sub-team focused on managing and scaling the organization's GPU compute platform, which supports cutting-edge AI safety research. You will be responsible for the technical direction and roadmap of the platform, ensuring its performance, fault tolerance, and security, especially as AI agents increasingly utilize the cluster for experiments. This position requires a blend of technical expertise and leadership, with at least half of your time dedicated to hands-on engineering work. You will also be responsible for hiring and growing the infrastructure team, setting priorities, and providing mentorship.

$225k - $325k

We are seeking a Senior Technical Support Engineer with deep expertise in Ethernet networking and AI infrastructure. In this customer-facing role, you will own complex technical investigations and critical escalations involving large-scale data center and AI environments. You will serve as a trusted technical advisor to strategic customers, combining strong troubleshooting skills with excellent communication, crisis management, and technical leadership. You will collaborate closely with Engineering, Product Management, Marketing, and Support teams to resolve issues and improve our products and support practices.

Salary not listed

FAR.AI is seeking a Senior Software Engineer to join their Foundations team, which is responsible for the institute's compute platform, research tools, and workflow automation. This role will focus on accelerating AI safety research by managing and scaling the GPU cluster fleet, which includes bare-metal and managed Kubernetes environments. The engineer will work directly with researchers and other engineers to ensure large-scale experiments are performant and fault-tolerant, contributing to the core infrastructure that enables cutting-edge AI safety advancements.

$150k - $275k

Luma is seeking a Software Engineer to own the serving of their AI models. This role involves integrating new model architectures into the inference engine, scaling deployments across thousands of machines, and optimizing the utilization of expensive GPU fleets while meeting internal service level objectives (SLOs). The work focuses on large-scale inference systems, including scheduling, fleet management, deployment pipelines, and ensuring reliability across diverse clusters and hardware providers. This position is ideal for a strong systems engineer comfortable with model serving and Kubernetes at scale, rather than those seeking pure modeling work.

Salary not listed

Luma is seeking a Research Scientist / Engineer to build and optimize the distributed systems that train large-scale multimodal models across thousands of GPUs. This role focuses on advanced parallelism, training stability, and utilization across massive clusters, requiring deep expertise in distributed PyTorch and CUDA. The goal is to provide researchers with reliable, efficient, and scalable infrastructure for innovation.

Salary not listed

Anthropic

Anthropic's Environments organization is responsible for building and maintaining the infrastructure that enhances Claude's capabilities through reinforcement learning. This includes developing frameworks for researchers to build environments and the underlying infrastructure for running them, with a core mission to productionize research. The role involves embedding with research teams to understand their workflows and designing frameworks and APIs that accelerate their progress, ensuring these systems are understandable, ownable, and maintainable. A significant part of the role also includes ensuring the health, maintainability, monitoring, and ease of triage for production RL runs. The ideal candidate will possess deep Python expertise, a strong sense of API and framework design, and an intuitive understanding of complex system failures, particularly silent ones. Experience with stateful distributed systems like workflow engines or actor frameworks is a plus. The role requires comfort in navigating and abstracting complex research codebases, incrementally improving them while research continues, and leveraging AI tools for development with a focus on verification.

$405k - $605k

Tenstorrent is seeking a Workload Performance Optimization Engineer to enhance the performance of software workloads on next-generation RISC-V platforms. This role involves optimizing modern datacenter and agentic AI workloads across various programming languages like Java, Python, and Rust. You will identify performance bottlenecks and develop optimizations at the software, compiler, runtime, and hardware levels to improve throughput, latency, and efficiency. The position bridges software runtimes, compilers, CPU microarchitecture, and RISC-V silicon, requiring profiling of real-world applications, investigating memory and concurrency, and leveraging RISC-V capabilities. You will also contribute to AI-assisted development and automated optimization workflows, directly impacting the RISC-V software ecosystem and future CPU architectures.

$100k - $500k

Code Metal

Code Metal is seeking a Staff AI Platform Engineer to architect and build the foundational AI enablement stack that internal engineering teams rely on. This role involves designing and operating production systems for GPU inference serving, model gateways, agent harnesses, context management, and observability. While primarily an engineering role focused on building and operating production systems, it also requires solid data science and AI research fundamentals to work closely with the AI Research team and conduct experiments when necessary. The platform is built to product standards, ensuring reliability and scalability for AI-driven code transpilation, modernization, optimization, modeling, and simulation solutions.

$205k - $250k

Runpod

Runpod is seeking a systems-focused Cloud Infrastructure Security Engineer to safeguard its bare-metal and virtualized GPU cloud infrastructure. This role is crucial for ensuring the security, multi-tenant isolation, and integrity of the underlying GPU cloud infrastructure. The ideal candidate will have deep knowledge of Linux systems, kernel internals, hypervisors, and containerization, focusing on preventing tenant breakouts, securing GPU hardware, and building resilient infrastructure for AI and machine learning workloads. You will operate with an attacker's mindset, identifying and patching vulnerabilities at the lowest levels of the stack.

$152k - $175k

Waabi is a leader in Physical AI, developing technology for autonomous transportation. We are seeking a Senior/Staff ML Ops Engineer to build and evolve our training infrastructure on Kubernetes, focusing on GPU scheduling, autoscaling, and multi-node distributed jobs. You will shape the developer-facing surface of our tools, shorten the development loop for ML engineers, and evangelize best-in-class tooling. This role involves strengthening our data and artifact layer, turning one-off scripts into durable, tested libraries, and making experiments legible through robust experiment tracking and model registries. You will also ship CI/CD for models, build comprehensive observability across the ML stack, and treat documentation and onboarding as a core product surface. Driving adoption through user empathy and prototyping is key, alongside ensuring platform reliability and building secure guardrails.

Salary not listed

MatX is building custom silicon and co-designing hardware/software to make the world's best AI models run with unprecedented efficiency. The AI tooling team plays a crucial role across the entire stack, developing agent harnesses and shared workflows that optimize general-purpose coding agents for specific layers. This position focuses on building the infrastructure that enhances AI agent judgment by providing them with necessary context and verification capabilities. You will work on making agents first-class users of our infrastructure, building secure execution environments, and developing a company-wide agent knowledge base. This is an opportunity to have a significant impact at a growing company where your work will directly influence every engineer using AI agents.

$160k - $275k

AssemblyAI

AssemblyAI is seeking a Software Engineer to bridge the gap between cutting-edge AI research and production-ready products. You will join a dynamic engineering team responsible for transforming new model capabilities from initial concepts into robust, well-documented APIs that handle production traffic at scale. This role involves designing and launching customer-facing APIs, scaling inference infrastructure, and collaborating with researchers and customers to ensure our products meet user needs and drive future development.

$190k - $225k

Cerebras

Cerebras Systems is seeking a Software Engineer to design and build the core software powering their engineering infrastructure. This role involves working on Python frameworks, orchestration systems, distributed execution, scheduling, test infrastructure, and developer tooling. The ideal candidate enjoys dissecting unfamiliar code, understanding system interdependencies, tackling complex debugging challenges, and improving underlying designs rather than implementing quick fixes. The team values strong software engineering fundamentals, independent problem-solving, and sound systems thinking.

Salary not listed