AI Infrastructure Engineer Jobs in San Francisco

Salient

Salient is seeking a Telephony Infrastructure Engineer to take ownership of the voice platform that powers our AI agents. This role involves managing the SIP edge, Asterisk, voice audio and dialing layers, and call servers. You will collaborate with AI agent engineers, telephony specialists, customer success teams, and carriers to ensure the platform's reliability, safety, and scalability for future growth. This is a hands-on position where you will complete ongoing projects, enhance platform stability, and prepare it to handle significantly increased call volumes. Additionally, you will design and implement customer telephony integrations, working towards making these processes more self-serve.

$200k - $300k

Together AI

Together AI is building the AI Native Cloud, an end-to-end platform for the full generative AI lifecycle, combining the fastest LLM inference engine with state-of-the-art GPU cloud infrastructure. The Together Cloud team builds the flagship IaaS product that provides high-performance, AI-ready GPU clusters through a self-serve cloud console, along with the virtualized infrastructure layer powering Together's inference, RL, and fine-tuning products. As a Staff Software Engineer focusing on AI Compute, you will set technical direction for and build major components of the next generation AI cloud platform – a highly available, global cloud infrastructure with cutting-edge virtualization of the latest ML hardware. This is an architect-and-build role where you will set the architecture for automated bootstrapping of GPU data centers, high-performance virtualization, and fault-tolerant decentralized control planes, and be a key owner of the hardest parts, in the code as well as the design. Your designs will span the IaaS layer up to the global management plane that schedules capacity across all data centers. At this level, the job is as much leverage as code: the standards you set and the engineers you grow decide how fast the rest of Together Cloud ships.

$260k - $300k

Together AI

Together AI is building the AI Native Cloud, an end-to-end platform for the full generative AI lifecycle, combining a fast LLM inference engine with state-of-the-art GPU cloud infrastructure. The Together Cloud team builds the flagship IaaS product that provides high-performance, AI-ready GPU clusters through a self-serve cloud console, along with the virtualized infrastructure layer powering Together's inference, RL, and fine-tuning products. As a Senior Software Engineer focusing on AI Compute, you will build and own major components of the next generation AI cloud platform – a highly available, global cloud infrastructure with cutting-edge virtualization of the latest ML hardware. This virtualized computing platform powers our own SaaS products and serves external cloud customers through self-serve offerings. The hard problems in rapidly scaling heterogeneous GPU fleets are software problems that we solve by building global and in-DC services, Kubernetes operators, and high-performance SDN libraries, forking hypervisors and Linux kernels, and building infra tailored to inference and fine-tuning. You'll own massive greenfield projects across their full lifecycle – scoping the problem, designing the system, and building it through to GA launch.

$220k - $270k

Together AI

Together AI is building the AI Acceleration Cloud, an end-to-end platform for the full generative AI lifecycle. The AI Infrastructure team is at the forefront of building and scaling the foundational systems that power our generative AI platform. The storage and observability team is crucial for designing, implementing, and maintaining robust distributed storage solutions, ensuring seamless data access and management. They are also responsible for developing comprehensive observability platforms, providing critical insights into system performance and GPU utilization, and proactively identifying and resolving issues.

$200k - $280k

Together AI

Together AI is building the AI Acceleration Cloud, an end-to-end platform for the full generative AI lifecycle. As a Senior Backend Engineer, you will play a key role in building the next generation AI cloud platform – a highly available, global, blazing-fast cloud infrastructure that virtualizes cutting-edge ML hardware and enables state-of-the-art ML practitioners with self-serve AI cloud services. This platform serves both internal StaaS products and external cloud customers across dozens of data centers worldwide.

$160k - $230k

Together AI

Together AI is building the Inference Platform to deliver advanced generative AI models, powering multi-tenant serverless workloads and dedicated endpoints. This platform enables developers, enterprises, and researchers to utilize the latest LLMs and multimodal models at scale. The role focuses on optimizing latency and maximizing the utilization of tens of thousands of GPUs. You will collaborate directly with research teams to productionize frontier models and engage with the open-source community, contributing to and leveraging projects like SGLang and vLLM to enhance inference performance and efficiency.

$200k - $290k

Together AI

As a Systems Research Engineer specialized in GPU Programming, you will play a crucial role in developing and optimizing GPU-accelerated kernels and algorithms for ML/AI applications. You will co-design GPU kernels and model architecture to enhance the performance and efficiency of our AI systems, and contribute to the co-design of efficient GPU architectures and programming models. Your research skills will be vital in staying up-to-date with the latest advancements in GPU programming techniques, ensuring that our AI infrastructure remains at the forefront of innovation.

$200k - $300k

Together AI

At Together AI, you will build and operate one of the world's largest GPU fleets for frontier model training and inference. This role is for engineers who excel at building systems, automating processes, and solving problems at a massive scale. If you prefer writing software over manual operations and aim to build infrastructure that autonomously manages tens of thousands of GPUs, we encourage you to apply.

$190k - $270k

MatX is building the compute platform for AGI, developing vertically integrated full-stack solutions from silicon to systems. We are seeking a hands-on engineering lead to manage, mentor, and grow a team of kernel engineers while setting technical direction and executing. This role involves actively contributing to the design and implementation of high-performance compute kernels for specialized AI hardware, partnering with ML teams to maximize output and improve diagnostics, and collaborating with hardware teams to co-design next-generation AI architectures.

$160k - $275k

Anthropic

The Safeguards ML Infra team is responsible for designing, building, and operating the production infrastructure that powers Claude's safety systems. This role is central to ensuring safeguards are properly configured and deployed for model launches, and owning the off-cycle deployment of new safety classifiers. The team ensures safety on the token generation path and manages the operational aspects of deploying these systems into production, including standing up safeguards for new model launches and deploying new safety classifiers. Every frontier model release goes through this team, which configures, verifies, and rolls out safeguards across all Claude platforms and leads incident response when issues arise. The goal is to automate manual checks into continuous validation and evolve launch runbooks into robust tooling.

$320k - $485k

Anthropic

Anthropic is at the forefront of AI research, dedicated to developing safe, ethical, and powerful artificial intelligence. Our mission is to ensure that transformative AI systems are aligned with human interests. We are seeking a Staff level Engineer to join our Pre-training team, responsible for developing the next generation of large language models. In this role, you will work at the intersection of cutting-edge research and practical engineering, contributing to the development of safe, steerable, and trustworthy AI systems.

$500k - $850k

Modal is building the next-generation infrastructure layer for AI, enabling companies to serve low-latency inference, fine-tune models, and access production-ready sandboxes at scale. Our platform provides instant GPU access, sub-second container starts, and native storage, supporting category-defining customers. We are seeking a leader to own the direction and execution of our LLM inference platform, a critical component of our offering that delivers frontier performance with best-in-class elasticity and developer experience. This is a hands-on leadership role where you will split your time between technical contributions, product shaping, and people management, setting direction, removing blockers, and fostering a strong engineering culture.

$300k - $350k

OpenAI

We are seeking an AI Systems Engineer to join our pAGI Infra team, which builds and operates the systems essential for large-scale model training and evaluation. This role involves owning projects from identifying bottlenecks and designing solutions to deployment and operation, focusing on distributed systems engineering, performance optimization, and collaboration with researchers. You will directly contribute to improving the speed and reliability of research by potentially building shared grading services, optimizing resource allocation, or integrating new training stacks into production.

$266k - $500k

Ambient.ai

Ambient.ai is seeking an AI Infrastructure Software Engineer to design, build, and optimize the AI infrastructure powering their real-time intelligence platform. This role involves working on systems to run state-of-the-art deep learning models, including computer vision, large language models (LLMs), large vision models (LVMs), and multimodal AI systems, across vast amounts of video data in real time. You will focus on scaling infrastructure for inference, evaluation, and continuous model improvement, partnering closely with research scientists and product engineering teams to bring AI advancements into production.

$168k - $205k

OpenAI

We are seeking a Software Engineer to develop the research infrastructure and tooling that will enable OpenAI models to design silicon. This role involves transforming chip-design workflows into robust environments for reinforcement learning and evaluation, and simplifying the process for researchers to conduct experiments and iterate on new ideas. You will bridge software engineering, tool integration, and open research challenges, leveraging strong coding fundamentals, clear technical judgment, and independent execution. While prior chip-design experience is beneficial, domain knowledge can be acquired alongside our hardware specialists.

$266k - $468k

Fireworks is seeking PhD graduates in Computer Science, Computer Engineering, or Electrical Engineering to join their Systems Infrastructure team. This role is for individuals who want to apply their research ideas to real-world, large-scale AI infrastructure. You will be responsible for designing and building the core systems that power Fireworks' AI platform, including schedulers, storage systems, and networks, ensuring efficient utilization of hardware and low inference latency. You will work on challenging problems related to compute placement, efficient data movement, and high-performance networking for AI workloads, with the opportunity to see your work ship in a production environment.

$200k - $230k

NVIDIA's Site Reliability Engineering (SRE) team is seeking a Principal SRE to shape the technical direction of the company's AI Platform Runtime. This role involves leading reliability engineering initiatives across multiple organizations, solving complex enterprise-scale problems, and establishing architectural standards for critical AI-powered services. The Principal SRE will also drive the adoption of AI agents and skills to enhance platform operations and developer productivity. The team values intellectual curiosity, diversity, collaboration, and continuous learning through blameless analysis, encouraging bold thinking and impactful improvements.

Salary not listed

Harvey is transforming legal and professional services by combining agentic AI, an enterprise-grade platform, and deep domain expertise. This role offers a unique opportunity to help build a generational company at an inflection point, with strong product-market fit and investor support. The team operates with intensity, ownership, and a commitment to excellence, valuing decisiveness, simplicity, and continuous improvement. If you are driven to do your best work alongside like-minded individuals, Harvey is the place to be.

$231k - $340k

Salesforce is seeking an experienced Senior Engineering Manager to lead the Foundations team, which is responsible for the core infrastructure and platform that all other engineering teams build upon. This role involves growing the team from 7 to 12 engineers, driving improvements in delivery, reliability, and technical direction. The ideal candidate will combine deep expertise in platform and infrastructure with strong people leadership, operational rigor, and a proactive approach. This is a high-visibility role focused on shaping the reliability, scalability, and velocity of the entire engineering organization within a high-growth, AI-focused environment.

$173k - $260k

Cerebras

Cerebras Systems is seeking a Senior Software Engineer to build and operate the platform layer behind their engineering infrastructure. This role focuses on designing, building, and maintaining CI/CD systems, Kubernetes-based platforms, and deployment automation. You will work on cloud and on-premises infrastructure, developer environments, artifact management, and observability, ensuring these systems are reliable, scalable, and easy to operate. This is an engineering-focused role where you will automate repetitive tasks, debug complex issues across system boundaries, and drive durable software and platform improvements.

Salary not listed

Anthropic

Our Inference team is responsible for building and maintaining the critical systems that serve Claude to millions of users worldwide. We bring Claude to life by serving our models via the industry’s largest compute-agnostic inference deployments. We are responsible for the entire stack from intelligent request routing to fleet-wide orchestration across diverse AI accelerators. The team has a dual mandate: maximizing compute efficiency to reliably serve our explosive customer growth, while enabling breakthrough research by giving our scientists the high-performance inference infrastructure they need to develop next-generation models. We tackle complex, distributed systems challenges across multiple accelerator families and emerging AI hardware running in multiple cloud platforms.

$320k - $485k

Perplexity AI

Perplexity is seeking a Technical Program Manager to act as the central point of contact between our model providers, engineering, and product teams, driving the advancement of our core inference platform. This role is crucial for managing one of the industry's highest-throughput inference stacks, serving diverse traffic across a dynamic portfolio of models. You will operate at the nexus of product, engineering, and finance, coordinating the smooth integration of new models and capacity while executing the roadmap for the inference platform itself. The ideal candidate possesses strong technical acumen, excels at coordinating diverse teams and external partners with competing deadlines, and is motivated by establishing operating models for functions with limited precedent.

$170k - $265k

Anthropic

Anthropic is building reliable, interpretable, and steerable AI systems to be safe and beneficial for users and society. This role leads a team of engineers focused on the critical control plane for Anthropic's inference fleet. This involves making key decisions about request routing and capacity allocation to meet throughput, reliability, and latency goals. The team designs placement and load-balancing algorithms, builds quantitative models for demand and performance, and optimizes latency across various system boundaries. The Engineering Manager will guide a team of ML platform, infrastructure, and distributed systems engineers, ensuring the efficiency, reliability, and evolution of the entire request-to-model path.

$405k - $625k