AI Infrastructure Engineer Jobs in San Francisco

Baseten

Baseten is seeking a Software Engineer to join the Agent Infra team, focusing on building and enhancing Agent Drive, a distributed filesystem critical for agent sandboxes. This role involves developing a storage solution that supports thousands of concurrent sandbox mounts with strict data isolation and consistency. The work leverages advanced storage techniques like tiered caching, replication, and content-addressed data, with a focus on Linux filesystem internals. The team operates with a high degree of ownership, shipping features rapidly and contributing to the core infrastructure that powers leading AI companies.

$160k - $330k

Together AI

We are seeking a reliability-minded software engineer to manage the production operations of our Kubernetes-native control plane, which provisions and runs our GPU inference fleet. This platform team develops the manifest-driven API and controllers that enable the inference team to define their needs, such as cluster configurations, model deployments, and capacity adjustments, and ensures these are reconciled. Your role will be to maintain the reliability of this system in production, addressing issues like InfiniBand problems during node communication, diagnosing faults in hero-model and LoRA serving that generic health checks miss, and enhancing automated self-healing to permanently resolve recurring failure modes. The primary goal is to ensure the decoupling remains effective even during failures and to maintain fleet efficiency without introducing new instabilities. You will be responsible for inference serving SLOs and error budgets, leading incident response, and implementing systemic fixes based on postmortem analyses. Additionally, you will contribute to the reliability aspects of capacity efficiency, recognizing that misconfigured defragmentation or bin-packing can pose production risks. We value candidates with hands-on production reliability experience in large-scale distributed or GPU-serving systems, proficiency in troubleshooting InfiniBand/RDMA and Kubernetes controller/CRD-based platforms, and a software engineering approach to GPU infrastructure and its supporting tools.

$160k - $280k

Figma is seeking infrastructure and platform-minded engineers to join our AI Platform team. This team builds the core frameworks, abstractions, and systems that power AI features across Figma, enabling new capabilities for AI product teams and improving performance, reliability, and technical quality. You will contribute to agent infrastructure, context retrieval & ranking platforms, and core AI services to accelerate critical company-wide AI initiatives. This role involves supporting end-to-end AI feature development by designing, building, and maintaining scalable, reliable, and performant systems, collaborating with cross-functional teams, and shaping platform strategy and best practices.

$153k - $376k

Baseten

Baseten powers mission-critical inference for leading AI companies by uniting applied AI research, flexible infrastructure, and seamless developer tooling. We enable companies at the frontier of AI to bring cutting-edge models into production. The Agent Infra team builds a custom networking stack for our sandboxes, designed for the scale of numerous isolated machines that constantly suspend, resume, and exchange data. This role focuses on building a user-space networking stack using VPP, enabling multi-tenant virtual private networking for agents and services.

$160k - $330k

OpenAI

OpenAI's Forward Deployed Engineering team partners with customers to turn research breakthroughs into production systems, operating at the intersection of customer delivery and core platform development. As a Forward Deployed Engineer for AI Infra, you will lead complex deployments of frontier models in production, embedding with customers where model performance and urgent delivery are critical. You will work directly with customer engineers, researchers, and product engineers on optimization problems across semiconductors and AI infrastructure, writing and debugging software, running experiments, and improving tools. Your role involves defining problems, structuring delivery, and shipping fast, with a focus on inference performance and chip-design optimization, ultimately contributing improvements to OpenAI's platform.

$185k - $300k

SambaNova

The SambaStack team is seeking an Engineering Director to lead a software engineering team while remaining a hands-on technical contributor. You will own the growth and effectiveness of your team, the technical quality of their output, and the roadmap for the inference serving platform. You will continue to design systems and write code on the platform's hardest problems, contributing directly as a software engineer on critical platform initiatives.

$245k - $325k

Baseten

Baseten powers mission-critical inference for leading AI companies, enabling them to bring cutting-edge models into production. We are building the platform engineers use to ship AI products, recently securing $1.5B in Series F funding. The Agent Infra team is responsible for the core virtualization layer that powers our agent sandboxes. This layer provides each agent with its own isolated microVM, capable of running code, calling tools, and maintaining state across long tasks. We are building a cloud infrastructure from the ground up, focusing on multi-tenancy and deep hardware integration.

$165k - $330k

Glean is seeking a Software Engineer to join our Platform Infrastructure team, which focuses on customer setup and deployment, particularly in complex cloud, hosting, and security environments. You will help build the systems that bootstrap cloud accounts and projects, wire authentication, and transform deployment workflows into reliable, repeatable productized experiences for customers. This role is ideal for an engineer who enjoys cloud infrastructure, networking, infrastructure as code, and solving real-world deployment problems with clean software abstractions.

$200k - $270k

Together AI

In this role, you will operate, scale, and optimize multi-petabyte storage systems purpose-built for the world’s largest AI training and inference workloads. You’ll manage and scale high-performance parallel filesystems and object stores, evaluate and integrate cutting-edge technologies, and solve complex engineering challenges of operating at extreme throughput, low-latency data paths, and massive cluster-scale storage operations. You will also build Kubernetes-native storage operators and self-service platforms that provide automated provisioning, strict multi-tenancy, performance isolation, and quota enforcement at cluster scale.

$250k - $330k

Glean is seeking a Senior Site Reliability Engineer to enhance the reliability, scalability, and operational excellence of its production services. This role involves designing, building, and operating resilient cloud infrastructure, improving service reliability against defined SLOs, and assisting teams in responding to production issues through effective observability and incident management practices. You will provide technical expertise for system and architectural decisions and collaborate with engineers across the organization to deliver dependable systems. Glean operates in a hybrid cloud environment and faces complex distributed-systems challenges due to its rapid growth. Your work will involve using software engineering and SRE skills to automate tasks, build tools, and improve deployment, monitoring, and on-call workflows, ultimately helping Glean scale and provide a reliable customer experience.

$200k - $260k

Baseten

Baseten is seeking a Backend/Distributed Systems Engineer to join their Product Platform team. This role is central to Baseten's mission of powering mission-critical AI inference for leading AI companies. You will work on key areas such as commercialization, billing, partnerships, developer productivity, agentic development, and the observability platform. The position offers the opportunity to design and build scalable systems, own core platforms, and contribute to the company's growth at the forefront of AI development.

$165k - $330k

Scale AI

Scale is at the forefront of the AI revolution, building the Generative AI Data Engine and other products that power the world's most advanced LLMs. The Platform Engineering team is foundational to these efforts, responsible for designing and developing shared platforms, architecting core cloud infrastructure, and redefining software development processes. This role offers exposure to the cutting edge of AI development across various sectors, from startups to governments. You will drive the design and implementation of critical platforms, collaborate with cross-functional teams, and proactively improve engineering practices. This is an opportunity to shape the future of AI infrastructure and contribute to some of the most important work in how humanity interacts with AI.

$180k - $225k

Databricks

Databricks is seeking a Senior Staff Software Engineer to work on the AI Runtime system, which powers the world's largest AI platform. This serverless product enables companies to train their own models using enterprise data, similar to how frontier labs operate. The role involves tackling complex technical challenges to advance AI Runtime's architecture, reliability, and developer experience for both humans and AI agents. You will contribute to shaping the long-term technical direction of AI training infrastructure, anticipating industry trends, and optimizing training efficiency and cost across various model architectures and GPU accelerators.

$229k - $343k

Scale AI

Scale Generative AI Platform (SGP) is seeking a skilled engineer to join its team and contribute to building and scaling its core infrastructure. This role involves implementing solutions across multiple cloud providers (GCP, Azure, AWS) for clients in regulated industries such as healthcare, telecom, finance, and retail. The ideal candidate will possess a strong foundation in software engineering principles and experience with large-scale distributed systems.

$179k - $224k

Relace

Relace is building a globally distributed inference cloud, competing on price, speed, and reliability. We process over 800 billion tokens daily and have experienced significant growth, backed by a $23 million Series A funding round. Our team is small, offering substantial responsibility and growth opportunities. We develop our own stack, running on Kubernetes across multiple clouds, where performance directly impacts business success. This role focuses on the lowest level of our stack, optimizing code that runs directly on GPUs to maximize work per second and dollar, shrink models for cost-efficiency without sacrificing quality, and eliminate GPU idle time.

Salary not listed

Lila Sciences is developing the infrastructure for autonomous science, creating AI systems that can design and execute experiments in physical labs across life sciences, chemistry, and materials science. Their platform is designed for reliability and scale, catering to external customers who demand security, auditability, and operational rigor. This role focuses on building foundational enterprise systems to support large pharmaceutical and research organizations, enhancing capabilities like multi-tenancy, identity management, granular permissions, audit infrastructure, and billing. The position involves pushing the boundaries with agentic AI-powered systems for self-service tenant management, intelligent customer observability, and automated compliance workflows, alongside developing an AI-native external API.

$192k - $288k

SambaNova

SambaNova is seeking an Inference Systems Performance Architect to lead the end-to-end performance strategy for their AI inference platform. This role involves defining and driving technical direction in workload capture, benchmarking, modeling, and simulation. You will be instrumental in understanding how requests move through the system, from tokenization to fabric communication, and how deployments meet customer service level objectives. The work focuses on creating reproducible representations of real and agentic traffic, developing what-if capabilities for future configurations, and tackling complex challenges in heterogeneous inference environments.

$245k - $325k

Pallet is building AI Agents to transform the physical economy, a massive global industry. We are a well-funded, high-growth startup with significant revenue growth and a mission to increase the efficiency of the global supply chain by automating manual workflows. Our platform provides end-to-end visibility and control, and our AI Agents integrate directly with customer systems to execute requests in real time. This role is for a Senior Member of Technical Staff, Infrastructure Engineer, focused on solving hard scaling and reliability problems to enable the next order of magnitude growth for Pallet's AI-powered supply chain solutions.

$200k - $265k

Anthropic

The Interpretability team at Anthropic is dedicated to understanding how modern language models work, believing that a deep, mechanistic understanding is crucial for making advanced AI systems safe and steerable. This role involves building and maintaining the specialized infrastructure that powers interpretability research, akin to performing 'neuroscience' on neural networks. The work spans the entire lifecycle of a production language model, from pretraining and inference to performance optimization, and directly contributes to safety audits on frontier models.

$315k - $560k

Glean is seeking a Backend/Infrastructure Engineer to build and evolve the Storage layer that powers a highly available, performant, secure, and cost-effective suite of storage options for all data at Glean. You will own the storage systems that handle Glean’s most sensitive data, from enterprise documents and permission hierarchies to user behavioral signals and real-time query path data. Your work will directly enable organizations at scale to discover and act on their critical knowledge, supporting millions of searches, chat interactions, and agent workflows. You’ll deliver robust uptime guarantees, autoscaling, and reliable alerting/monitoring for a key component that runs across multiple cloud providers. As part of the Storage team, you’ll be responsible for a critical piece of Glean’s infrastructure, working closely with several other teams to support their workloads. You’ll also work across the stack to abstract cloud differences, reduce human intervention through self-healing systems, and drive real business impact by enabling Glean’s ever-growing scale.

$200k - $300k

Glean is seeking a Software Engineer, Developer Productivity to enhance how our engineers build, test, and ship software. This role involves optimizing our multi-language codebase, which is powered by Bazel and runs on GitHub Actions with cloud remote execution. You will be instrumental in designing and refining build systems, CI/CD pipelines, and developer tools. A key aspect of this position will be enabling engineers to effectively utilize AI-powered coding and productivity tools, ultimately reducing friction in daily workflows, scaling our infrastructure, and accelerating the entire engineering team's progress with confidence.

$140k - $220k

Glean is seeking software engineers to contribute to building a leading search and assistant product for the workplace. Engineers will work across the technology stack, focusing on areas such as generative AI, Retrieval Augmented Generation (RAG), query and document understanding, domain-adapted language models, natural language question-answering, and evaluation. The role involves close interaction with customers to understand their challenges and applying the most effective tools to solve them.

$175k - $270k

OpenAI

OpenAI's Hardware organization is developing next-generation AI-native silicon and integrated systems to power frontier models. This role focuses on defining and building the CI/CD systems essential for qualifying these advanced models and the software stack that runs on OpenAI's custom AI accelerators. The work ensures that correctness, performance, and stability are measurable for every code change, from initial commits to production releases. This position involves model workloads, accelerator software, systems infrastructure, and enhancing developer productivity by establishing scalable CI/CD and GitOps practices, designing robust regression and benchmarking pipelines, and collaborating with various engineering teams to optimize feedback loops.

$318k - $485k

Together AI

We are seeking a software engineer to develop a Kubernetes-native control plane for provisioning and managing our GPU inference fleet. This role involves designing a manifest-driven API that allows the inference team to declare their needs, such as clusters, model deployments, or capacity adjustments. Our controllers will handle the underlying reconciliation, provider/runtime selection, and lifecycle management, abstracting away the complexities of the serving stack, scheduler, or hardware pool. You will also build systems to ensure fleet efficiency, including defragmentation, rebalancing logic for consolidating workloads, and scheduling/bin-packing improvements to maximize GPU utilization without compromising latency. The core objective is to decouple platform users from operational complexities while optimizing hardware capacity. This includes developing controllers, reconciliation loops, and self-service interfaces (API/CLI) for seamless operation, alongside event-driven systems for health monitoring, remediation, and utilization management, all designed to operate without human intervention. A product-oriented mindset is crucial, with a focus on building internal platforms and APIs that enhance the developer experience for consuming engineering teams. This role operates under a "you build it, you own it" philosophy, encompassing both software delivery and production support.

$160k - $280k