Distributed Systems Jobs

OpenAI

We are seeking a software engineer to build and evolve our classification and evaluation platform. You will develop services, online and offline pipelines, and tools that help researchers and data scientists turn an idea into a working measurement and iterate on it quickly. This role combines software and data engineering with hands-on production ownership. You will shape the architecture, deliver new capabilities, improve inference efficiency, and keep the system dependable as workloads and requirements evolve. Strong candidates will bring deep software engineering expertise and an interest in building a technical product in close partnership with its users.

$347k - $445k

Anthropic

Anthropic is dedicated to building AI systems that are safe, reliable, and steerable, aiming to benefit both users and society. This role is central to advancing Reinforcement Learning (RL), the core technology enabling Claude's reasoning, coding, and autonomous capabilities. The RL Scaling team focuses on how RL performance, stability, and learning efficiency evolve with increasing model size, episode length, and compute power. You will be instrumental in developing and scaling next-generation RL algorithms and architectures, from initial research to large-scale deployment, while deeply understanding and addressing the challenges that arise at scale. This position bridges research and engineering, requiring you to build the systems that accelerate iteration and experimentation, and to tackle complex, unsolved problems in Anthropic's largest RL runs.

$500k - $850k

Anthropic

Reinforcement learning is crucial for training AI models to reason, write code, and act autonomously. At scale, this process involves a demanding distributed system where training, sampling, and environment execution run concurrently across a large fleet of accelerators and hosts. This system must continuously exchange data and maintain progress despite hardware failures, shifting loads, and evolving research. The efficiency of this system directly impacts learning outcomes and the speed of research iteration. As a Research Engineer on the Distributed Systems team within RL Engineering, you will focus on the most critical bottlenecks within this system. This could involve areas like scheduling, data movement, sandboxed environment execution, storage, networking, fault tolerance, autoscaling, or observability. The role seeks generalist engineers capable of reasoning from first principles across different system layers and prioritizing impactful problems over familiar ones. The ideal candidate will have experience building and operating large distributed systems, a commitment to detail, and a desire to apply their expertise to a complex, heterogeneous, and rapidly changing workload.

$500k - $850k

Datadog is seeking a customer-focused, deeply technical Distinguished Architect to join our Product Solutions Architecture team. In this role, you will act as a technical multiplier for leading AI labs and AI-native companies, bridging the gap between their infrastructure aspirations and Datadog’s technology roadmap. You will ensure our platform natively solves the unique observability challenges of training and deploying foundational models at scale. We operate as a hybrid workplace to foster collaboration and creativity while ensuring work-life harmony.

$330k - $440k

Joining NVIDIA's DGX Cloud AI Efficiency Team means advancing the performance, efficiency, and resiliency of large-scale AI workloads. You will help AI researchers and platform teams understand end-to-end behavior across GPUs, networking, storage, and software stacks. Your work will shape scalable DGX Cloud systems, turn complex measurements into prioritized engineering decisions, and continuously raise the performance and reliability of AI workloads. Join a technically diverse team of infrastructure experts to unlock more efficient AI at scale.

$224k - $431k

Salesforce is seeking a Senior AI Engineer to join the Agentforce Operations team, a high-velocity 'startup-within-Salesforce' focused on revolutionizing the public sector supply chain with an AI-powered platform. This role offers the unique opportunity to build a '0 to 1' product in a rapidly growing vertical, working alongside a founding team with deep startup roots but backed by the global scale of Salesforce. You will be instrumental in shaping both the product and the engineering team, partnering with product leaders and sector experts to deliver trusted AI systems for mission-critical public-sector workflows. The role involves building AI features tailored to public-sector customers, designing systems with security, compliance, reliability, and operational constraints at their core, and operating with wide autonomy to raise the bar for technical quality and mentorship.

$149k - $224k

NVIDIA is seeking a Senior Software Engineer to lead the bring-up, triage, benchmarking, analysis, and optimization of distributed training and inference workloads across NVIDIA GPU platforms at the largest scales. In this role, you will set technical direction for communication libraries, model frameworks, and inference/training stacks to ensure state-of-the-art LLM workloads run efficiently and reliably at scale. You will lead deep performance and reliability investigations on multi-GPU and multi-node deployments, define how new platforms are benchmarked and qualified, and build the resilience and failure-attribution capabilities essential for large clusters. This is a hands-on, senior individual-contributor role for an engineer experienced at the intersection of deep learning systems, GPU performance, distributed computing, and large-scale operations.

$184k - $357k

Joining NVIDIA's DGX Cloud AI Efficiency Team means contributing to the infrastructure that powers our innovative AI research. This team focuses on developing tools for optimizing efficiency and resiliency of AI workloads, including pre-training, post-training, and inference. Our objective is to deliver a stable, scalable environment for AI researchers, providing them with the necessary resources and scale to foster innovation. As a senior DGX Cloud AI Infrastructure software engineer, you will have the opportunity to work on innovative technologies that power the future of AI and data science and be part of a dynamic, diverse, and supportive team that values learning and growth.

$184k - $357k

Anthropic

Anthropic is building reliable, interpretable, and steerable AI systems to be safe and beneficial for users and society. The Cloud Inference team is responsible for scaling and optimizing Claude to serve massive audiences across AWS, GCP, Azure, and other cloud providers. This involves owning the end-to-end product on each platform, from API integration and intelligent request routing to inference execution, capacity management, and operations. Engineers on this team are highly leveraged, driving multiple revenue streams while optimizing compute resources. The role requires product-minded backend engineers who can navigate platform differences, design cross-provider services and abstractions, and make architectural decisions for reliability and cost-effectiveness at massive scale. The work directly contributes to increasing service scale, accelerating the launch of new models and features, and ensuring LLMs meet rigorous safety, performance, and security standards.

$320k - $485k

Anthropic

The Cloud Inference team is responsible for scaling and optimizing Claude to serve massive audiences of developers and enterprise companies across major cloud platforms like AWS, GCP, and Azure. Within this team, the model & inference launch team focuses on the validation pipeline for inference servers and load balancers. This critical infrastructure work ensures that every inference change, including new model launches and performance improvements, is deployed with correctness, performance, and reliability. The goal is to make validation fast, cost-effective, and trustworthy, directly impacting how quickly advanced models and features reach users on all cloud platforms and how efficiently compute resources are utilized.

$320k - $485k

Mercor is a leading AI data company that organizes human intelligence to power the AI economy. We build the layer between human expertise and frontier models, enabling millions of domain experts to train AI models at scale. As a Tech Lead for the Applied AI Backend Systems, you will own critical services within our platform, focusing on designing data models, building APIs and services, and developing data pipelines that ensure reliable, fast, and observable operations. This is a hands-on build role where you will take ambiguous problems, make design decisions, ship to production, and mentor other engineers.

$250k - $500k

Mercor is a leading AI data company building the layer between human expertise and frontier models, organizing human intelligence to power the AI economy. The Applied AI organization focuses on building systems that transform human expertise into training data for advanced models, creating task pipelines, expert workflows, and evaluation infrastructure. This role involves owning and developing backend services within this stack, focusing on data models, APIs, and data pipelines that ensure correctness, speed, and observability as the platform scales. It's a hands-on build role where you'll take ownership from problem scoping through to production deployment and ongoing maintenance.

$220k - $500k

Together AI

Together AI is building the AI Acceleration Cloud, an end-to-end platform for the full generative AI lifecycle. As a Senior Backend Engineer, you will play a key role in building the next generation AI cloud platform – a highly available, global, blazing-fast cloud infrastructure that virtualizes cutting-edge ML hardware and enables state-of-the-art ML practitioners with self-serve AI cloud services. This platform serves both internal StaaS products and external cloud customers across dozens of data centers worldwide.

$160k - $230k

Together AI

Together AI is building the Inference Platform to deliver advanced generative AI models, powering multi-tenant serverless workloads and dedicated endpoints. This platform enables developers, enterprises, and researchers to utilize the latest LLMs and multimodal models at scale. The role focuses on optimizing latency and maximizing the utilization of tens of thousands of GPUs. You will collaborate directly with research teams to productionize frontier models and engage with the open-source community, contributing to and leveraging projects like SGLang and vLLM to enhance inference performance and efficiency.

$200k - $290k

Together AI

Together AI runs one of the largest GPU fleets globally, and the Infra Agent Systems team is responsible for developing the software that powers and automates this infrastructure. The team builds production AI agents designed to diagnose hardware failures, investigate incidents, correlate signals across the fleet, and automate operational workflows. Alongside these agents, they are building the core platform, which includes knowledge graphs, retrieval systems, orchestration frameworks, and developer tooling. This role offers the unique opportunity to work at the intersection of AI agents, distributed systems, infrastructure, and automation, tackling complex engineering challenges with significant real-world impact.

$250k - $300k

Together AI

Together.ai is seeking a Senior Software Engineer for the Customer Insights team. This team is responsible for the customer-facing visibility layer of Together's Cloud, enabling users to understand, trace, and act on their environment with confidence. You will tackle ambiguous problems, define technical direction, and influence architecture beyond your immediate scope. This role offers significant autonomy, with a collaborative team environment for review and learning.

$180k - $250k

WRITER is seeking an exceptional software engineer to join their rapidly evolving team. In this pivotal role, you will be at the forefront of expanding human capacity by building the next generation of AI-powered solutions that transform how leading enterprises operate. You will dive deep into developing a state-of-the-art platform that leverages cutting-edge generative AI technologies, from large language models to sophisticated agentic workflows, delivering seamless, scalable, and secure applications that redefine enterprise productivity. This is an unparalleled opportunity to make a tangible impact, shaping the future of AI and contributing to a product that’s changing how the world works. This role is hybrid, based out of their San Francisco, New York City, or Seattle hubs.

$132k - $240k

Databricks

Databricks is seeking a Staff Engineer to join the Customer Engagement and Docs platform team, focusing on enhancing customer support through AI and Large Language Models (LLMs). This role operates at the intersection of Product development and Machine Learning, aiming to help customers resolve issues faster by providing precise solutions or facilitating seamless escalations. You will gain a deep understanding of user behavior, identify problem drivers, and leverage ML and LLMs to analyze complex technical use cases within an enterprise environment, ultimately guiding them towards resolution.

$190k - $260k

Databricks

As a staff software engineer for GenAI inference, you will lead the architecture, development, and optimization of the inference engine that powers Databricks Foundation Model API. You’ll bridge research advances and production demands, ensuring high throughput, low latency, and robust scaling. Your work will encompass the full GenAI inference stack: kernels, runtimes, orchestration, memory, and integration with frameworks and orchestration systems.

$191k - $233k

Databricks

Databricks is seeking a Staff Software Engineer to architect and deliver the company's AI-native vertical workflow platform, Customer Experience Intelligence (CXI). This platform is designed to orchestrate AI agents, humans, and enterprise tools into end-to-end operational workflows, initially focusing on customer support and expanding to domains like IT, Security, Legal, and Marketing. The role involves building a scalable, multi-tenant agent platform with a strong emphasis on reliability, observability, and enterprise-grade features.

$164k - $226k