AI Infrastructure Engineer Jobs at Startups

Code Metal

Code Metal is seeking a Staff AI Platform Engineer to architect and build the foundational AI enablement stack that internal engineering teams rely on. This role involves designing and operating production systems for GPU inference serving, model gateways, agent harnesses, context management, and observability. While primarily an engineering role focused on building and operating production systems, it also requires solid data science and AI research fundamentals to work closely with the AI Research team and conduct experiments when necessary. The platform is built to product standards, ensuring reliability and scalability for AI-driven code transpilation, modernization, optimization, modeling, and simulation solutions.

$205k - $250k

Sierra

Sierra is seeking a Software Engineer for its Inference team to build the systems that power their AI agents. This role focuses on making foundation models fast, reliable, and efficient at scale. You will define the inference architecture, working on systems for serving, routing, capacity management, and optimization for latency, reliability, and cost. This is a systems-first role at the intersection of distributed infrastructure and AI, requiring strong distributed systems fundamentals and experience with complex systems problems.

$230k - $390k

Runpod

Runpod is seeking a systems-focused Cloud Infrastructure Security Engineer to safeguard its bare-metal and virtualized GPU cloud infrastructure. This role is crucial for ensuring the security, multi-tenant isolation, and integrity of the underlying GPU cloud infrastructure. The ideal candidate will have deep knowledge of Linux systems, kernel internals, hypervisors, and containerization, focusing on preventing tenant breakouts, securing GPU hardware, and building resilient infrastructure for AI and machine learning workloads. You will operate with an attacker's mindset, identifying and patching vulnerabilities at the lowest levels of the stack.

$152k - $175k

Together AI

Together AI is seeking a Staff Platform Infrastructure Engineer to lead service and platform infrastructure strategy across the engineering organization. This is a hands-on technical leadership role with the opportunity to establish a new team and define the standard for how services are built and operated. You will provide a stable, scalable service infrastructure foundation for mission-critical product and platform teams, focusing on reliability, repeatability, and ease of adoption. You will own and evolve crucial shared infrastructure, roll out new patterns and platform capabilities, and ensure services are built on company-wide foundations.

$240k - $280k

Join us as we build the next generation of enterprise search infrastructure powering AI agents. Our mission is to evolve from traditional product-specific search systems into a composable, agent-native platform that enables agents and applications to discover, retrieve, reason over, and recommend information across heterogeneous enterprise data sources. As a Senior Staff Engineer, you will be the technical owner for foundational search infrastructure that serves as the backbone of our agentic platform. You will drive the architecture, scalability, and evolution of core indexing and retrieval systems while partnering closely with Search Ranking, ML, Platform, and AI teams to deliver production-grade search capabilities that power agent experiences across the company.

$191k - $334k

d-Matrix

d-Matrix is seeking a Sr Staff Site Reliability Engineer to join their AI Infrastructure team. This role is crucial for owning the infrastructure layer that powers all engineering teams and customers, encompassing colocation facilities, on-premises GPU clusters, cloud environments, and platform services. You will be responsible for reliability, automation, and observability across these diverse environments, managing systems end-to-end from provisioning to incident response. This is a hands-on position focused on building and operating real infrastructure, collaborating with hardware and software teams on CI/CD, QA, and HPC workloads, and supporting customer-facing deployments.

$175k - $265k

HeyGen is building AI applications including Avatar IV, Photo Avatar, Interactive Avatar, and Video Translation. We are looking for a Software Engineer focused on GPU performance to make the systems behind these experiences faster and more efficient. You will work across model execution and inference infrastructure, using profiling and measurement to improve latency, throughput, and GPU cost. This role is a fit for an engineer who enjoys understanding how software uses the hardware beneath it.

Salary not listed

Salient

Salient is seeking a Telephony Infrastructure Engineer to take ownership of the voice platform that powers our AI agents. This role involves managing the SIP edge, Asterisk, voice audio and dialing layers, and call servers. You will collaborate with AI agent engineers, telephony specialists, customer success teams, and carriers to ensure the platform's reliability, safety, and scalability for future growth. This is a hands-on position where you will complete ongoing projects, enhance platform stability, and prepare it to handle significantly increased call volumes. Additionally, you will design and implement customer telephony integrations, working towards making these processes more self-serve.

$200k - $300k

Together AI

Together AI is building the AI Native Cloud, an end-to-end platform for the full generative AI lifecycle, combining the fastest LLM inference engine with state-of-the-art GPU cloud infrastructure. The Together Cloud team builds the flagship IaaS product that provides high-performance, AI-ready GPU clusters through a self-serve cloud console, along with the virtualized infrastructure layer powering Together's inference, RL, and fine-tuning products. As a Staff Software Engineer focusing on AI Compute, you will set technical direction for and build major components of the next generation AI cloud platform – a highly available, global cloud infrastructure with cutting-edge virtualization of the latest ML hardware. This is an architect-and-build role where you will set the architecture for automated bootstrapping of GPU data centers, high-performance virtualization, and fault-tolerant decentralized control planes, and be a key owner of the hardest parts, in the code as well as the design. Your designs will span the IaaS layer up to the global management plane that schedules capacity across all data centers. At this level, the job is as much leverage as code: the standards you set and the engineers you grow decide how fast the rest of Together Cloud ships.

$260k - $300k

Together AI

Together AI is building the AI Native Cloud, an end-to-end platform for the full generative AI lifecycle, combining a fast LLM inference engine with state-of-the-art GPU cloud infrastructure. The Together Cloud team builds the flagship IaaS product that provides high-performance, AI-ready GPU clusters through a self-serve cloud console, along with the virtualized infrastructure layer powering Together's inference, RL, and fine-tuning products. As a Senior Software Engineer focusing on AI Compute, you will build and own major components of the next generation AI cloud platform – a highly available, global cloud infrastructure with cutting-edge virtualization of the latest ML hardware. This virtualized computing platform powers our own SaaS products and serves external cloud customers through self-serve offerings. The hard problems in rapidly scaling heterogeneous GPU fleets are software problems that we solve by building global and in-DC services, Kubernetes operators, and high-performance SDN libraries, forking hypervisors and Linux kernels, and building infra tailored to inference and fine-tuning. You'll own massive greenfield projects across their full lifecycle – scoping the problem, designing the system, and building it through to GA launch.

$220k - $270k

Together AI

Together AI is building the AI Acceleration Cloud, an end-to-end platform for the full generative AI lifecycle. The AI Infrastructure team is at the forefront of building and scaling the foundational systems that power our generative AI platform. The storage and observability team is crucial for designing, implementing, and maintaining robust distributed storage solutions, ensuring seamless data access and management. They are also responsible for developing comprehensive observability platforms, providing critical insights into system performance and GPU utilization, and proactively identifying and resolving issues.

$200k - $280k

Together AI

Together AI is building the AI Acceleration Cloud, an end-to-end platform for the full generative AI lifecycle. As a Senior Backend Engineer, you will play a key role in building the next generation AI cloud platform – a highly available, global, blazing-fast cloud infrastructure that virtualizes cutting-edge ML hardware and enables state-of-the-art ML practitioners with self-serve AI cloud services. This platform serves both internal StaaS products and external cloud customers across dozens of data centers worldwide.

$160k - $230k

Together AI

Together AI is building the Inference Platform to deliver advanced generative AI models, powering multi-tenant serverless workloads and dedicated endpoints. This platform enables developers, enterprises, and researchers to utilize the latest LLMs and multimodal models at scale. The role focuses on optimizing latency and maximizing the utilization of tens of thousands of GPUs. You will collaborate directly with research teams to productionize frontier models and engage with the open-source community, contributing to and leveraging projects like SGLang and vLLM to enhance inference performance and efficiency.

$200k - $290k

Together AI

As a Systems Research Engineer specialized in GPU Programming, you will play a crucial role in developing and optimizing GPU-accelerated kernels and algorithms for ML/AI applications. You will co-design GPU kernels and model architecture to enhance the performance and efficiency of our AI systems, and contribute to the co-design of efficient GPU architectures and programming models. Your research skills will be vital in staying up-to-date with the latest advancements in GPU programming techniques, ensuring that our AI infrastructure remains at the forefront of innovation.

$200k - $300k

Together AI

At Together AI, you will build and operate one of the world's largest GPU fleets for frontier model training and inference. This role is for engineers who excel at building systems, automating processes, and solving problems at a massive scale. If you prefer writing software over manual operations and aim to build infrastructure that autonomously manages tens of thousands of GPUs, we encourage you to apply.

$190k - $270k

Isomorphic Labs is seeking a Staff Software Engineer to join their Inference Platform team in London. This role is pivotal in ensuring the reliability and scalability of the company's AI infrastructure, which is used to build foundation models for drug discovery. You will play a key part in hardening systems and ensuring the AI infrastructure is stable, reliable, and can handle increasing data volumes and model sizes as the company grows. This is an opportunity to contribute to groundbreaking innovation in applying AI to cure disease within an interdisciplinary team.

Salary not listed

Together AI

As a Customer Support Engineer at Together AI, you will be the primary point of contact for customers building AI solutions, including training, fine-tuning, and inference. You will tackle complex technical challenges, provide expert solutions, and act as a product specialist. This role involves close collaboration with product and sales teams to enhance offerings and drive customer success in a fast-paced, innovative AI environment.

Salary not listed

Together AI

Together AI is seeking an experienced AI Infra (Systems Engineers) Lead/Manager to join our team in Amsterdam. You will be responsible for keeping all user-facing services and production systems running smoothly, blending operational discipline with software engineering principles. This role involves applying sound engineering practices and automation to our operating environments and codebase, with a specialization in systems like operating systems, storage, and networking, while focusing on availability, reliability, and scalability.

Salary not listed

Together AI

Together AI is seeking an AI Infrastructure Engineer (SRE) to ensure the smooth operation of user-facing services and production systems. This role combines the skills of a pragmatic operator and a software engineer, applying sound engineering principles, operational discipline, and automation to our operating environments and codebase. You will specialize in systems such as operating systems, storage subsystems, and networking, while implementing best practices for availability, reliability, and scalability, with interests in algorithms and distributed systems. Join a research-driven artificial intelligence company focused on advancing AI through open and transparent systems, aiming to significantly lower the cost of modern AI systems by co-designing software, hardware, algorithms, and models.

Salary not listed

MatX is building custom silicon and co-designing hardware/software to make the world's best AI models run with unprecedented efficiency. The AI tooling team plays a crucial role across the entire stack, developing agent harnesses and shared workflows that optimize general-purpose coding agents for specific layers. This position focuses on building the infrastructure that enhances AI agent judgment by providing them with necessary context and verification capabilities. You will work on making agents first-class users of our infrastructure, building secure execution environments, and developing a company-wide agent knowledge base. This is an opportunity to have a significant impact at a growing company where your work will directly influence every engineer using AI agents.

$160k - $275k

MatX is building the compute platform for AGI, developing vertically integrated full-stack solutions from silicon to systems. We are seeking a hands-on engineering lead to manage, mentor, and grow a team of kernel engineers while setting technical direction and executing. This role involves actively contributing to the design and implementation of high-performance compute kernels for specialized AI hardware, partnering with ML teams to maximize output and improve diagnostics, and collaborating with hardware teams to co-design next-generation AI architectures.

$160k - $275k

Ambient.ai

Ambient.ai is seeking an AI Infrastructure Software Engineer to design, build, and optimize the AI infrastructure powering their real-time intelligence platform. This role involves working on systems to run state-of-the-art deep learning models, including computer vision, large language models (LLMs), large vision models (LVMs), and multimodal AI systems, across vast amounts of video data in real time. You will focus on scaling infrastructure for inference, evaluation, and continuous model improvement, partnering closely with research scientists and product engineering teams to bring AI advancements into production.

$168k - $205k

Manifold is seeking a Staff Software Engineer to join its Core Platform team, a senior group responsible for the foundational services powering the company's data, compute, and workflow capabilities. In this hands-on, individual contributor role, you will work on compute infrastructure, workflow orchestration, data pipelines, platform APIs, and MCP services. You will support enterprise users and their AI agents analyzing large-scale genomics and life sciences datasets. The role offers an opportunity to shape the longer-term product architecture and contribute to core services critical for accelerating the delivery of life-changing medicines to patients.

$170k - $220k

Harvey is transforming legal and professional services by combining agentic AI, an enterprise-grade platform, and deep domain expertise. This role offers a unique opportunity to help build a generational company at an inflection point, with strong product-market fit and investor support. The team operates with intensity, ownership, and a commitment to excellence, valuing decisiveness, simplicity, and continuous improvement. If you are driven to do your best work alongside like-minded individuals, Harvey is the place to be.

$231k - $340k