Distributed Training Jobs

An applied research team within NVIDIA's Networking Systems & Software Architecture group is tackling complex AI infrastructure challenges. The team develops systems-level software to facilitate high-speed data transfer between GPUs, nodes, and storage, crucial for modern AI demands. This involves optimizing low-level transport, co-designing hardware and software, and creating communication frameworks that integrate with production AI stacks. The team's scope also extends to emerging areas like quantum computing interconnects. The Senior Architect will lead modules and projects from initial research scoping through to production code delivery, acting as a recognized expert to guide technical decisions, incorporate external ideas, and prototype novel solutions.

$224k - $431k

NVIDIA is seeking a dedicated Software Engineer, ML Performance to join our team in Santa Clara, CA. In this role, you will engage directly with the latest AI technologies and work on optimizing performance at an outstanding scale. Your expertise will ensure that our groundbreaking solutions operate flawlessly and continue to lead the industry. You will be immersed in an inclusive, supportive environment where everyone is inspired to do their best work. Come join the team and see how you can make a lasting impact on the world.

$224k - $357k

Anduril Industries is seeking a High Performance Computing (HPC) System Engineer to directly support sensitive programs by building and maintaining large-scale HPC infrastructure. This role offers the opportunity to work with leading engineers and cybersecurity professionals on cutting-edge systems deployed globally for national security missions. You will be instrumental in architecting, deploying, and managing advanced GPU infrastructure, ensuring the seamless operation of critical compute environments.

$147k - $196k

Figure is an AI robotics company developing autonomous general-purpose humanoid robots with the goal of shipping robots with human-level intelligence for home and commercial markets. The Helix team is responsible for developing the core AI systems that power humanoid autonomy. We are seeking a Helix AI Engineer, Video Pretraining to lead the development of large-scale video foundation models trained on diverse real-world and robot-collected data. This role focuses on pretraining models that learn from raw video, capturing motion, interaction, and temporal structure to enable downstream capabilities in perception, prediction, and embodied reasoning.

$200k - $400k

Figure is an AI robotics company developing autonomous general-purpose humanoid robots with the goal of shipping human-level intelligence. The Helix team is seeking an experienced AI Training Performance Engineer to elevate model training. This role focuses on enhancing distributed training frameworks for large-scale model training, optimizing GPU kernels, evaluating different accelerator types, and co-designing models for maximum hardware utilization.

$200k - $400k

The Voyage AI team at MongoDB is building state-of-the-art embedding models and rerankers for efficient unstructured data search and retrieval, crucial for RAG, recommendation systems, and semantic search. This team, comprised of AI researchers from top universities, was recently acquired by MongoDB and is integrating its advanced embedding models with MongoDB's data platform to create powerful end-to-end solutions. We are seeking a Senior Research Scientist to join this dynamic team and contribute to the development of next-generation AI models, tackling complex challenges at the intersection of machine learning research and the practical deployment of large neural networks.

$126k - $248k

CoreWeave is seeking a Senior Engineer to join its Benchmarking & Performance team. This role will focus on building performance insights and observability systems for AI infrastructure, covering everything from individual GPUs and interconnects to distributed training and inference workloads. You will be responsible for detecting, diagnosing, and surfacing performance signals across global data centers, transforming raw telemetry into actionable insights for engineers, product teams, and executives. This is a hands-on role focused on building observability tooling that provides near real-time answers about infrastructure performance, rather than a traditional data engineering or BI position.

$182k - $242k

The CRAFT (Capabilities, Routing, Adaptation & Fine-Tuning) team at CoreWeave is focused on developing tools that enable AI agents to learn from experience, a crucial step towards autonomous long-task execution. The team has already released innovative projects like ART for RL, RULER for reward functions, and Serverless RL for infrastructure management. They are now tackling remaining challenges in agent training, such as reducing human labor in environment creation and improving data efficiency. This role offers a significant opportunity to contribute to hard-tech advancements in a rapidly growing market, with the potential to shape the future of self-improving AI agents.

$182k - $242k

CoreWeave is seeking a Principal Engineer to lead its Benchmarking & Performance team. This role will be responsible for a global performance data warehouse, ingesting, storing, transforming, and analyzing performance events across all data centers. You will also play a key part in industry-leading performance benchmarking publications, working closely with NVIDIA and the open-source community to demonstrate CoreWeave's performance and reliability leadership in AI.

$206k - $333k

You'll join the Grok Voice Model team as a software engineer. You'll design, build, and run the software systems behind Grok Voice, which delivers natural, low-latency, multilingual spoken interactions across devices and in real time. The team owns the full engineering stack behind the model, from large-scale data processing to distributed training to production deployment.

$150k - $450k

Perplexity is seeking a Member of Technical Staff to join their GPU Cluster Infrastructure team. This role will own the underlying GPU infrastructure that powers Perplexity's AI inference requests, which serve hundreds of millions of queries monthly. The goal is to abstract away the complexity of GPU provisioning, cluster configuration, and multi-cloud management into a unified, self-serve platform for both training and inference workloads. You will be responsible for the entire lifecycle of the GPU fleet, ensuring reliability, efficient capacity utilization, and seamless operation across different cloud providers.

$250k - $485k

NVIDIA is seeking a Senior Software Engineer to join our DSX team. You will build and operate tooling that verifies AI infrastructure readiness for production, focusing on cluster readiness and certification software. This includes working with the open-source Cluster Readiness Engine to validate GPU clusters across Kubernetes and Slurm, measure training and inference performance, and produce trustworthy evidence for customers and partners. You will collaborate with leading AI companies and cloud providers to ensure their infrastructure performs as designed for critical workloads.

$184k - $357k

NVIDIA is seeking an NCX Senior Engineer to join our DSX team, collaborating closely with strategic customers to implement and enhance groundbreaking AI workloads. You will deliver hands-on technical assistance for advanced AI deployments, intricate distributed systems, and ensure customers realize efficient performance from NVIDIA's AI platform across varied environments. We partner with the world's most innovative AI companies to address their most challenging technical problems. In this role, you will develop innovative solutions that advance AI infrastructure capabilities and directly influence customer success with breakthrough AI initiatives.

$184k - $357k

Accenture

As a hands-on AI Infrastructure Architect, you will be responsible for designing and optimizing the AI and machine learning infrastructure that powers real-world applications. You will work alongside senior architects and engineers, applying your skills in coding, testing, configuring, deploying, monitoring, and troubleshooting AI systems and their underlying infrastructure. This role involves architecting and optimizing infrastructure components, writing and reviewing code and deployment scripts, designing and tuning compute resources like GPU clusters and distributed training environments, and deploying AI systems into production. You will also build and optimize data pipelines, monitor system health across InfraOps and MLOps, and troubleshoot complex issues. Additionally, you will mentor junior engineers, contribute to architectural decisions, and help establish best practices, making meaningful contributions to infrastructure that enables AI-driven business outcomes.

Anduril Industries is seeking a Senior AI Infrastructure Engineer to lead the vision, execution, and long-term stability of how the company trains with GPUs at scale. This role involves taking ownership of cluster robustness, ensuring high-performance GPU systems are highly available, fault-tolerant, and resilient for ML platform and research teams. The primary focus is on logical stability and automated resilience, building self-healing mechanisms, tuning networking, and optimizing scheduling to ensure massive-scale training infrastructure runs seamlessly and scales efficiently.

$166k - $220k

Anduril Industries is seeking a Senior AI Infrastructure Engineer to lead the vision, execution, and long-term stability of how the company trains with GPUs at scale. This role involves taking ownership of cluster robustness, ensuring high-performance GPU systems are highly available, fault-tolerant, and resilient for ML platform and research teams. The primary focus is on logical stability and automated resilience, building self-healing mechanisms, tuning networking, and optimizing scheduling to ensure massive-scale training infrastructure runs seamlessly and scales efficiently.

$166k - $220k

Anduril Industries is a defense technology company focused on transforming military capabilities with advanced technology. We are seeking a Machine Learning Engineer to join the Air Dominance & Strike team, which designs, builds, and flies autonomous air vehicles. This role is crucial for accelerating our design cycles by applying the latest research in physics ML to critical bottlenecks. You will own the entire surrogate modeling stack, from architectures and training infrastructure to data pipelines and user tooling, directly impacting hardware decisions for our air vehicle programs.

$220k - $292k

Anduril Industries is a defense technology company focused on transforming military capabilities with advanced technology. Our core Lattice platform integrates sensors to provide real-time situational awareness. As a Software Engineer on the ML Infrastructure team, you will develop the core systems and infrastructure that perception engineers use to field autonomous edge systems. This includes developing data management systems for perception and autonomy, model training/orchestration systems, and an internal perception platform for data querying, model training, and evaluation.

$167k - $222k

The RL infrastructure team is looking for an engineer to help develop our RL training framework. This role involves designing and implementing the systems that support all RL workloads at SpaceXSI, from small-scale experiments to production training runs. You will be responsible for profiling, debugging, and optimizing the end-to-end training performance, as well as improving the scalability and observability of the RL stack.

$180k - $440k

Mercor is seeking an AI Infrastructure Engineer to build and manage the GPU and accelerator platform that powers our AI workloads. This role involves overseeing clusters, schedulers, training and inference serving, and the observability and cost data associated with fleet usage. You will write production code daily and own systems end-to-end, from design to on-call responsibilities. Mercor is a leading AI data company, building the layer between human expertise and frontier models, with a mission to organize human intelligence to power the AI economy. We are a profitable Series C company valued at $10 billion.

$200k - $400k