AI Infrastructure Engineer Jobs at Startups

Together AI

We are seeking a software engineer to develop a Kubernetes-native control plane for managing our GPU inference fleet. This role involves designing a manifest-driven API that allows the inference team to declare their needs for clusters, model deployments, or capacity changes. Our controllers will handle the underlying reconciliation, provider/runtime selection, and lifecycle management, abstracting away the complexities of the serving stack, scheduler, and hardware. You will also build systems to optimize fleet efficiency, including defragmentation, rebalancing logic for consolidating workloads, and scheduling/bin-packing improvements to maximize GPU utilization without impacting latency. The primary goal is to decouple users of the platform from operational complexities while maximizing hardware capacity. This includes building controllers, reconciliation loops, and self-service APIs/CLIs, along with event-driven systems for health, remediation, and utilization, all designed to operate without human intervention. A product mindset is crucial, focusing on the developer experience of internal platforms and APIs, with a 'build it, own it' philosophy that includes production operation and support.

Salary not listed

Vectara

Vectara is seeking a Platform Engineer to own the infrastructure that runs our deploy-anywhere platform. This role involves managing Kubernetes clusters for ML inference at scale, CI/CD pipelines, infrastructure as code, and the observability stack. You will be hands-on, writing Helm charts and Terraform, debugging issues, and shipping backend service features. The position requires deploying across AWS, GCP, and on-premises environments, including air-gapped setups, and participating in an on-call rotation to support enterprise customers.

Salary not listed

Invisible is an Applied AI research company that builds platform infrastructure to enable AI in production at scale. We help companies unify data, orchestrate agents, and deploy workflow applications. Our platform supports clients like Grubhub and Thomson Reuters, creating a feedback loop between enterprise deployment and model training. We are recognized for our predictive analytics platform and are focused on making AI work effectively in real-world scenarios.

Salary not listed

Zencoder.ai is building and orchestrating AI agents designed to handle real-world tasks like coding, research, and operations. What began as developer tooling has evolved into a collaborative platform for humans and AI agents working together on knowledge-based tasks. This role focuses on 'AI harness engineering,' ensuring the fundamental systems like CI/CD, testing, infrastructure as code, observability, and secure delivery are robust enough for AI agents to operate productively. The goal is to enhance developer experience and platform engineering, making workflows smoother for both humans and AI, and to integrate security seamlessly into the development process, addressing the evolving threat landscape posed by AI agents.

Salary not listed

Vectara

Vectara is seeking a Senior Platform Software Engineer to join their team. This role focuses on developing and managing the core infrastructure for their enterprise RAG and Agentic AI Platform, ensuring high availability, scalability, and automation. You will work on Infrastructure as Code (IaC) and Helm charts, design and develop platform features, and contribute to the Forward Deployed Engineering function by supporting customer applications from POCs to production. The role involves collaborating with Field Engineering teams, deploying and optimizing ML/NLP workloads, and contributing to architecture and infrastructure decisions as the platform scales. You will also be part of an on-call rotation and contribute to improving system reliability and developer experience.

Salary not listed

Relace

Relace is building a globally distributed inference cloud, competing on price, speed, and reliability. We process over 800 billion tokens daily and have experienced significant growth, backed by a $23 million Series A funding round. Our team is small, offering substantial responsibility and growth opportunities. We develop our own stack, running on Kubernetes across multiple clouds, where performance directly impacts business success. This role focuses on the lowest level of our stack, optimizing code that runs directly on GPUs to maximize work per second and dollar, shrink models for cost-efficiency without sacrificing quality, and eliminate GPU idle time.

Salary not listed

Pallet is building AI Agents to transform the physical economy, a massive global industry. We are a well-funded, high-growth startup with significant revenue growth and a mission to increase the efficiency of the global supply chain by automating manual workflows. Our platform provides end-to-end visibility and control, and our AI Agents integrate directly with customer systems to execute requests in real time. This role is for a Senior Member of Technical Staff, Infrastructure Engineer, focused on solving hard scaling and reliability problems to enable the next order of magnitude growth for Pallet's AI-powered supply chain solutions.

$200k - $265k

LangChain is seeking an Engineering Manager to lead the team responsible for building LangSmith, our observability and evaluation platform for LLM applications. This role involves defining the technical direction, nurturing a talented engineering team, and collaborating with product and design to deliver features that empower developers to build and deploy trustworthy AI systems. You will play a key part in shaping the future of AI observability at scale, contributing to a company that has raised $125M and is experiencing rapid growth.

$200k - $250k

Together AI

We are seeking a software engineer to develop a Kubernetes-native control plane for provisioning and managing our GPU inference fleet. This role involves designing a manifest-driven API that allows the inference team to declare their needs, such as clusters, model deployments, or capacity adjustments. Our controllers will handle the underlying reconciliation, provider/runtime selection, and lifecycle management, abstracting away the complexities of the serving stack, scheduler, or hardware pool. You will also build systems to ensure fleet efficiency, including defragmentation, rebalancing logic for consolidating workloads, and scheduling/bin-packing improvements to maximize GPU utilization without compromising latency. The core objective is to decouple platform users from operational complexities while optimizing hardware capacity. This includes developing controllers, reconciliation loops, and self-service interfaces (API/CLI) for seamless operation, alongside event-driven systems for health monitoring, remediation, and utilization management, all designed to operate without human intervention. A product-oriented mindset is crucial, with a focus on building internal platforms and APIs that enhance the developer experience for consuming engineering teams. This role operates under a "you build it, you own it" philosophy, encompassing both software delivery and production support.

$160k - $280k

Prime Intellect

Prime Intellect is building the open superintelligence stack, providing infrastructure for AI labs. Their platform, Lab, unifies compute, environments, evaluations, and training/deployment for post-training at scale, including SFT, RL, and agent workflows. They are developing open-source models for long-horizon tasks and the platform their research team uses. The company has raised significant funding and is looking for individuals passionate about frontier research, infrastructure, and go-to-market strategies in a nascent category.

$150k - $350k

Typeface

We help the world’s biggest brands move from brief to fully personalized campaigns — in days, not months. Founded by Abhay Parasnis and backed by Microsoft, GV, Salesforce, Lightspeed, Madrona and Menlo, we’re building category-defining technology at the intersection of creativity and AI with real impact. Join us to help shape the future of enterprise marketing. We're looking for a Staff Infrastructure Engineer to help lead core parts of our platform. You'll design, build, and scale the cloud infrastructure that powers our AI and GenAI workloads — ensuring reliability, security, and performance as we grow. This is your chance to shape the backbone of a platform trusted by Fortune 100 companies — building systems that are resilient, observable, and ready to scale with the next wave of AI.

$210k - $240k

d-Matrix

d-Matrix is at the forefront of software and hardware innovation, aiming to unleash the potential of generative AI. We foster a culture of respect, collaboration, humility, and direct communication, valuing diverse perspectives to create better solutions. We are seeking passionate, execution-driven individuals ready to tackle challenges and shape the future of AI. The Inference Cloud Engineering team is responsible for the deployment and orchestration technology behind d-Matrix’s inference services, including an internal cloud for model testing and a customer-facing cloud with self-service inference endpoints. This role will own the end-to-end testing and release process for this critical software, ensuring features are proven, releases are predictable, and rollbacks are safe, even with scarce and shared hardware resources.

$155k - $235k

You.com is building the AI Search Infrastructure that powers modern AI systems, aiming to create a trusted knowledge layer for agents, applications, and enterprises. Our platform combines proprietary vertical indexes with LLM-optimized retrieval systems. We are seeking a Senior Backend Engineer to own and scale the inference gateway, which is the critical layer that handles all model calls. This role involves routing, fallback, caching, metering, and enforcement within strict latency budgets, sitting between model serving and product development.

$200k - $250k

sarvam

Sarvam is building India's full-stack sovereign AI platform, focusing on research, models, infrastructure, and applications to make AI work for India. We partner with leading enterprises and public institutions, backed by top venture capital firms. We are seeking a Software Engineer to own major parts of our model APIs platform end-to-end. This platform handles millions of daily API calls for services like ASR, TTS, Vision, LLM, and Translate, including features like metering, billing, and rate-limiting. You will be responsible for the reliability, performance, and low latency of critical components, working closely with our Staff Engineer.

Salary not listed

Physical Intelligence

Physical Intelligence is building general-purpose AI for the physical world, developing foundation models and learning algorithms for robots and future physically-actuated devices. The Infrastructure team is central to this mission, building and operating the backbone for training state-of-the-art VLA models, large-scale simulation, and reliable deployment of intelligence across robot fleets. This role involves collaborating closely with researchers, robotics runtime, product, and platform engineers to ensure infrastructure scales from prototype to production-grade deployments.

Salary not listed

Code Metal

Code Metal is seeking a Senior Software Engineer for its AI Platform team. This role involves designing, building, and operating the foundational AI enablement stack that powers the company's AI-driven code solutions. You will contribute to critical components such as GPU inference serving, model gateways, agent harnesses, context engineering, observability, and AI experimentation management. While this is an internal platform, it is built to product standards. The position offers the opportunity to eventually own a specific area, like model serving or agentic infrastructure, while also contributing broadly across the stack. The role requires a blend of strong software engineering skills with solid data science and AI research fundamentals, including occasional experimental work to inform platform decisions.

$170k - $210k

Relevance AI is building the future of the AI workforce, empowering teams to delegate meaningful work to intelligent AI agents. Our platform enables the creation and management of agents that handle complex workflows and decisions, already used by industry leaders. We are backed by prominent investors and have secured Series B funding to accelerate growth. Headquartered in San Francisco and Sydney, we operate on a hybrid model focused on curiosity, collaboration, and execution. This role is crucial for building the factory that ships every feature, focusing on an end-to-end system where coding agents can pick up, build, test, review, and deploy tasks safely and at speed. You will own the developer experience, platform infrastructure, and reliability, shaping the foundations of our AI-native platform and multiplying the output of every engineer and agent.

Salary not listed

Luma is seeking a Software Engineer to own the serving of their AI models. This role involves integrating new model architectures into the inference engine, scaling deployments across thousands of machines, and optimizing the utilization of expensive GPU fleets while meeting internal service level objectives (SLOs). The work focuses on large-scale inference systems, including scheduling, fleet management, deployment pipelines, and ensuring reliability across diverse clusters and hardware providers. This position is ideal for a strong systems engineer comfortable with model serving and Kubernetes at scale, rather than those seeking pure modeling work.

$195k - $345k

Luma is seeking a Research Scientist / Engineer to build and optimize the distributed systems that train large-scale multimodal models across thousands of GPUs. This role focuses on advanced parallelism, training stability, and utilization across massive clusters, requiring deep expertise in distributed PyTorch and CUDA. The goal is to provide researchers with reliable, efficient, and scalable infrastructure for innovation.

$195k - $395k

Lovable is seeking an infrastructure engineer to build the data ingestion, storage, and query systems that power their agentic software creation platform. This role is crucial for ensuring the reliability of data for applications built on Lovable. The company is a fast-growing AI company based in Stockholm, with a culture of ownership, velocity, and collaboration. They are looking for individuals who are passionate about making a significant impact in the next era of software development.

Salary not listed

Tenstorrent is seeking a Workload Performance Optimization Engineer to enhance the performance of software workloads on next-generation RISC-V platforms. This role involves optimizing modern datacenter and agentic AI workloads across various programming languages like Java, Python, and Rust. You will identify performance bottlenecks and develop optimizations at the software, compiler, runtime, and hardware levels to improve throughput, latency, and efficiency. The position bridges software runtimes, compilers, CPU microarchitecture, and RISC-V silicon, requiring profiling of real-world applications, investigating memory and concurrency, and leveraging RISC-V capabilities. You will also contribute to AI-assisted development and automated optimization workflows, directly impacting the RISC-V software ecosystem and future CPU architectures.

$100k - $500k

sarvam

Sarvam is building the bedrock of Sovereign AI for India, developing a full-stack AI platform focused on research, models, infrastructure, and applications. This role owns the infrastructure for training foundational models at scale, involving deep systems work such as distributed training, parallelism strategies, GPU kernel optimization, and reliability engineering. The position demands a high level of expertise to identify and resolve complex issues that can significantly impact training timelines and costs.

Salary not listed

Join our Conversation Engine team to scale and optimize a cutting-edge Generative AI product that provides instant assistance to enterprise users. You will work at the core of Moveworks' Generative AI product, focusing on scaling and optimizing it for the enterprise solution space. We encourage innovative ideas to solve unsolved problems, collaborating closely with machine learning experts and cross-functional teams to rapidly iterate on new ideas and leverage user behavior data. Your challenge will be to scale and optimize our conversation engine to support hundreds of millions of users across multiple chat channels and use cases, enhancing our end-to-end product infrastructure with engineering quality and robustness.

$176k - $308k

Percepta

Percepta is seeking an AI Infrastructure Engineer to take ownership of the infrastructure, deployment, and operational reliability for Percepta's AI systems, including autonomous agents. This role involves hardening existing infrastructure, strengthening deployment pipelines, and defining new SRE practices for operating autonomous systems. You will be instrumental in bridging the gap between AI research and production, contributing to the foundational decisions that shape the future of agentic infrastructure.

$180k - $400k