AI Infrastructure Engineer Jobs at Startups

Supabase

Supabase is seeking a Software Engineer, AI Tooling Infrastructure to join their AI Tooling team. This role focuses on building and operating the backend services that power Supabase's AI products, such as the Supabase MCP server and services supporting agent skills and the dashboard assistant. The position involves ensuring these services are reliable, correctly authorized, and observable, handling the unique traffic patterns of AI clients and agents. It's a systems and infrastructure-focused role, requiring hands-on building and operation rather than just advisory. The ideal candidate thrives in fast-paced, asynchronous environments and is passionate about creating scalable developer tools.

Salary not listed

Perplexity AI

Perplexity is seeking software engineers to join our Training Infrastructure team. This team is responsible for large-scale RL and agent evaluation platforms that enable the company to train state-of-the-art models and optimize agent harnesses. As a full-stack engineer on this team, you will build distributed systems that orchestrate AI workloads across global-scale compute infrastructure, define novel user and agent interfaces for designing, managing, and launching experiments, and create tools that make experiment results legible to humans and agents alike. A strong candidate will possess both experience and passion for solving difficult problems at the intersection of AI, distributed systems, and research acceleration.

$220k - $405k

Netomi is seeking a hands-on DevOps Architect to design, build, and manage cloud infrastructure on AWS for their agentic AI platform. This role involves setting technical direction, writing infrastructure and automation code, and advising client stakeholders. The architect will also play a crucial part in leveraging AI tools to enhance development speed and quality while maintaining security standards. The company works with major global brands to enable AI-powered automation in customer experience.

Salary not listed

Fal

fal is building the generative media ecosystem for the next generation of AI products, providing the infrastructure, tools, and model access needed to move from idea to production at scale. We offer a unified platform for high-performance inference, orchestration, and observability, enabling new categories of AI-native products. As generative media reshapes industries, fal is becoming the foundation for ambitious teams. This role focuses on empowering fal's ML team by developing automation, infrastructure, and developer tooling to streamline the development, testing, and deployment of generative AI models. You will own and enhance CI/CD systems for a growing collection of ML models and inference pipelines, aiming to eliminate manual work, accelerate development cycles, and build reliable systems for confident model shipping and optimization.

$170k - $230k

Harvey is transforming legal and professional services by combining frontier agentic AI, an enterprise-grade platform, and deep domain expertise. This role is an opportunity to help build a generational company at an inflection point, with strong product-market fit and investor support. The team moves fast, takes ownership, and is deeply committed to the mission, operating with intensity and pushing for excellence. As a Member of Technical Staff, Infrastructure, you will build the systems that power how we adapt and improve leading open-weight models for legal work, focusing on scaling post-training efforts and owning the infrastructure behind distributed training, RL rollouts, agent environments, and internal tooling for data management, evals, and job tracking.

$230k - $340k

Turing

Turing is seeking a Senior DevOps & Infrastructure Engineer to join their global Infrastructure Engineering team. This role focuses on building and maintaining the cloud infrastructure and developer platform that powers the company's operations. You will be responsible for making systems more reliable, secure, scalable, and efficient as the company grows. This is a hands-on position for an engineer who thrives on automating manual tasks, solving complex production issues, and elevating operational excellence. You will collaborate closely with application and security teams, working across cloud infrastructure, reliability, developer tooling, and security. The role offers meaningful ownership within a globally distributed team, requiring regular overlap with U.S. Pacific Time working hours and participation in a global on-call rotation.

Salary not listed

Vectara

Vectara is seeking a Senior/Staff Platform Software Engineer to build and scale the core infrastructure powering their AI agent platform. This role involves designing robust services and scalable systems with a focus on automation, observability, reliability, and performance. You will work on AI-assisted coding, lead code and system design reviews, and mentor team members. The position requires designing, implementing, and maintaining scalable and secure backend services, driving infrastructure automation, CI/CD, and monitoring pipelines, and collaborating with product and research teams to deploy and optimize ML/NLP workloads. You will also lead initiatives to improve system reliability and scalability, contribute to architecture decisions, and champion best practices in code quality and DevOps.

Salary not listed

SpaceXAI is seeking an engineer to join our RL infrastructure team and help develop our evaluation infrastructure. This role involves building highly reliable, efficient, and user-friendly systems to run evaluations at scale. You will collaborate with modeling and evaluation teams to develop new evaluations, maintain signal quality, and create internal tooling for training next-generation models. A key part of this role is identifying and resolving performance bottlenecks across the entire evaluation infrastructure stack, including inference, capacity fleet management, and asynchronous evaluation orchestration.

$180k - $440k

SpaceXSI is building the high-performance inference platform that serves Grok to millions of users daily with exceptional speed and reliability. As a Member of Technical Staff - Inference, you will be instrumental in designing and optimizing large-scale model serving systems from the ground up. This includes owning distributed infrastructure components like global KV cache, continuous batching, load balancing, and auto-scaling, as well as delving into deep low-level optimizations such as GPU kernels, quantization, and speculative decoding. This is a high-impact role where your contributions will directly influence the speed and reliability of user interactions with Grok at a massive scale.

$180k - $440k

Figure is an AI robotics company developing autonomous general-purpose humanoid robots with the goal of shipping robots with human-level intelligence. These robots are engineered to perform a variety of tasks in home and commercial markets. We are seeking a Staff AI Inference & Acceleration Engineer to join our Platform Software team. This role will be responsible for the on-board inference architecture for our humanoid robots, acting as the technical authority on how AI workloads are mapped, optimized, and executed across the robot's compute hardware. The primary objective is to reduce power consumption and cost while meeting strict latency and reliability demands for a real-time autonomous system.

$180k - $275k

Turing

Turing's mission is to accelerate superintelligence to drive real economic progress. The company works with frontier AI labs to generate high-quality datasets, reinforcement learning environments, and frontier research benchmarks. Turing also partners with Fortune 500 enterprises to build and deploy end-to-end agentic AI systems within mission-critical workflows. This role involves leading the Turing Enterprise AI Platform technical team, which is being built out for a late 2025 SDK release and a Q1 2027 tenant onboarding. The goal is to grow the team to eight to ten members and collaborate with the Chief AI Architect on client-facing architecture and evaluation.

Salary not listed

SpaceXSI is building one of the world's largest AI supercomputers from the ground up. As part of the Compute Infrastructure team, you will own both the raw GPU supercomputer and the platform layer that runs on top of it. You will work across the full stack—from low-level GPU kernel optimizations and Linux kernel internals to massive-scale orchestration and virtualization—to make training and inference at SpaceXSI as fast, reliable, and scalable as possible. This is a broad, high-impact role that combines hardcore supercompute and compute infrastructure work. Your contributions will directly accelerate Grok's training speed and overall AI progress.

$180k - $440k

Perplexity AI

Perplexity is seeking a Member of Technical Staff to join their GPU Cluster Infrastructure team. This role will own the underlying GPU infrastructure that powers Perplexity's AI inference requests, which serve hundreds of millions of queries monthly. The goal is to abstract away the complexity of GPU provisioning, cluster configuration, and multi-cloud management into a unified, self-serve platform for both training and inference workloads. You will be responsible for the entire lifecycle of the GPU fleet, ensuring reliability, efficient capacity utilization, and seamless operation across different cloud providers.

$250k - $485k

Figure is an AI robotics company developing autonomous general-purpose humanoid robots with the goal of shipping them with human-level intelligence for home and commercial markets. The Helix team is looking for an experienced Data Infrastructure Engineer to advance the AI data infrastructure. This role involves building tools and software components for robot data management, including offloading, storage, manipulation, and access, while managing on-premise and cloud resources and providing production support. The ideal candidate will have experience building tools and infrastructure for large-scale autonomous or deep learning systems.

$150k - $400k

Together AI

We are seeking a software engineer with a strong focus on reliability to manage the production operations of our Kubernetes-native control plane, which provisions and runs our GPU inference fleet. This platform team develops the API and controllers that enable the inference team to define their needs, such as clusters, model deployments, or capacity changes, and ensures these are reconciled. Your role will be to maintain this functionality in the real world by troubleshooting issues like InfiniBand problems, diagnosing serving faults, and enhancing our automated self-healing systems to permanently resolve recurring failures. The primary goal is to ensure the decoupling remains effective even during disruptions and to maintain fleet efficiency without introducing new instabilities. You will be responsible for inference serving SLOs and error budgets, leading incident response, and implementing lessons learned from postmortems into system changes. Additionally, you will address the reliability aspects of capacity efficiency, recognizing that misconfigured defragmentation or bin-packing can pose production risks. We value candidates with hands-on production reliability experience in large-scale distributed or GPU-serving systems, proficiency in troubleshooting InfiniBand/RDMA and Kubernetes controller-based platforms, and a software engineering approach to GPU infrastructure.

$250k - $320k

Together AI

We are seeking a reliability-minded software engineer to manage the production operations of our Kubernetes-native control plane, which provisions and runs our GPU inference fleet. This platform team develops the manifest-driven API and controllers that enable the inference team to define their needs, such as cluster configurations, model deployments, and capacity adjustments, and ensures these are reconciled. Your role will be to maintain the reliability of this system in production, addressing issues like InfiniBand problems during node communication, diagnosing faults in hero-model and LoRA serving that generic health checks miss, and enhancing automated self-healing to permanently resolve recurring failure modes. The primary goal is to ensure the decoupling remains effective even during failures and to maintain fleet efficiency without introducing new instabilities. You will be responsible for inference serving SLOs and error budgets, leading incident response, and implementing systemic fixes based on postmortem analyses. Additionally, you will contribute to the reliability aspects of capacity efficiency, recognizing that misconfigured defragmentation or bin-packing can pose production risks. We value candidates with hands-on production reliability experience in large-scale distributed or GPU-serving systems, proficiency in troubleshooting InfiniBand/RDMA and Kubernetes controller/CRD-based platforms, and a software engineering approach to GPU infrastructure and its supporting tools.

$160k - $280k

Turing

Turing is seeking a Senior Software Engineer to join their AI Evaluation Infrastructure team. This role focuses on building the core infrastructure that frontier AI labs use to evaluate coding agents and language models. You will be responsible for designing and implementing features across the entire stack, from data models and background jobs to user-facing interfaces. The ideal candidate is an experienced, independent engineer who can translate loosely defined requests into robust, tested, and reliable solutions. This position offers the opportunity to work on cutting-edge AI evaluation tools and contribute to the advancement of AI capabilities.

$170k - $200k

Turing

Enterprise AI is building the platform that enables reliable deployment of AI systems across enterprise environments. As a Platform Engineer, you will build and operate the core of Turing’s Enterprise AI platform, including platform services, the model gateway, multi-tenant infrastructure, CI/CD, infrastructure as code, and observability. The central challenge is creating repeatable deployments across environments Turing may not control, including client cloud accounts, dedicated instances, and sovereign installations. This role requires hands-on software engineering and operational ownership, with a focus on building production software and contributing deeply to platform services.

Salary not listed

Together AI

At Together AI, you’ll build and operate one of the world’s largest GPU fleets used for frontier model training and inference. This role is for engineers who love building systems, automating everything, and solving problems at massive scale. If you enjoy writing software more than clicking dashboards, obsess over eliminating manual work, and want to build infrastructure that manages tens of thousands of GPUs autonomously, we’d love to talk.

Salary not listed

sarvam

Sarvam is building Indic-language AI for large-scale use, and the CPU Infrastructure team is responsible for all systems outside the GPU fleet. This includes managing Kubernetes across multiple cloud providers, the network infrastructure, identity and access management, observability, cost optimization, and the product delivery pipeline. The team operates on an end-to-end ownership model, where engineers are empowered to automate repetitive manual tasks by building scripts, pipelines, bots, or AI agents. As an AI company, Sarvam integrates AI agents into its own operations for tasks like planning infrastructure changes, triaging alerts, and handling routine issues, with engineers providing oversight. In this role, you will ensure the smooth operation of Sarvam's product infrastructure, handling daily tasks such as deployments, provisioning, access control, incident response, and cost management. A key expectation is to automate these operational duties. The ideal candidate possesses an instinct for identifying manual processes and building automated solutions, whether through scripts, pipeline steps, bots, or AI agents. As you reduce operational toil, your focus will shift towards platform and automation development. You will contribute to the infrastructure supporting a variety of AI products, including voice agents, coding agents, and API platforms.

Salary not listed

Together AI

We are seeking a software engineer to develop a Kubernetes-native control plane for managing our GPU inference fleet. This role involves designing a manifest-driven API that allows the inference team to declare their needs for clusters, model deployments, or capacity changes. Our controllers will handle the underlying reconciliation, provider/runtime selection, and lifecycle management, abstracting away the complexities of the serving stack, scheduler, and hardware pool. You will also build systems to optimize fleet efficiency, including defragmentation, rebalancing logic for consolidating workloads, and scheduling/bin-packing improvements to maximize GPU utilization without impacting latency. The primary goal is to decouple platform users from operational complexities while increasing usable capacity from existing hardware. This includes developing controllers, reconciliation loops, and self-service interfaces (API/CLI) for seamless decoupling, alongside event-driven systems for health monitoring, remediation, and utilization management without manual intervention. A product mindset is crucial, with experience building internal platforms or APIs for other engineering teams and a focus on developer experience and ownership of delivered software, including production support.

Salary not listed

Together AI

In this role, you will operate, scale, and optimize multi-petabyte storage systems purpose-built for the world’s largest AI training and inference workloads. You’ll manage and scale high-performance parallel filesystems and object stores, evaluate and integrate cutting-edge technologies, and solve complex engineering challenges of operating at extreme throughput, low-latency data paths, and massive cluster-scale storage operations. You will also build Kubernetes-native storage operators and self-service platforms that provide automated provisioning, strict multi-tenancy, performance isolation, and quota enforcement at cluster scale.

$250k - $330k

Cohere Health

As a DevOps and Site Reliability Engineer (SRE), you will work closely with application teams to automate and optimize infrastructure, and solve issues in an exciting, fast-paced environment. You will play a vital role in ensuring our systems are reliable, scalable, and high-performing. You will design, implement, and manage robust observability solutions across our cloud infrastructure and applications, comfortable working with metrics, logs, and traces to proactively detect, diagnose, and resolve issues.

Salary not listed

Together AI

As a Technical Support Engineer at a pioneering AI company, you will be the first line of defense supporting customers building training, fine-tuning, and inference solutions. You will address complex technical challenges, provide effective solutions, and act as a product expert. Collaborating with product and sales teams, you will contribute to the continuous improvement of offerings. This role is for a technically proficient individual passionate about AI and customer success, seeking to make a significant impact in a fast-paced, innovative environment.

$160k - $230k