New: upload your resume and get matched to AI jobs. Free to start →

Find your next AI engineering role.

276 open roles at 309 companies, from top AI labs and YC startups. Refreshed every day.

Rather not search? Upload your resume and see the jobs you fit, best first. Get matched →

276 open positions

Staff Software Engineer, Inference / Compute Infrastructure Engineering London or Amsterdam

2mo ago
Together AI

Together AI

We are seeking a software engineer to develop a Kubernetes-native control plane for managing our GPU inference fleet. This role involves designing a manifest-driven API that allows the inference team to declare their needs for clusters, model deployments, or capacity changes. Our controllers will handle the underlying reconciliation, provider/runtime selection, and lifecycle management, abstracting away the complexities of the serving stack, scheduler, and hardware pool. You will also build systems to optimize fleet efficiency, including defragmentation, rebalancing logic for consolidating workloads, and scheduling/bin-packing improvements to maximize GPU utilization without impacting latency. The primary goal is to decouple platform users from operational complexities while increasing usable capacity from existing hardware. This includes developing controllers, reconciliation loops, and self-service interfaces (API/CLI) for seamless decoupling, alongside event-driven systems for health monitoring, remediation, and utilization management without manual intervention. A product mindset is crucial, with experience building internal platforms or APIs for other engineering teams and a focus on developer experience and ownership of delivered software, including production support.

London & Amsterdam onsite Full-time
KubernetesPythonGo +5 more

[Junior / Senior / Staff] Software Engineer, Inference / Compute Infrastructure Engineering

3mo ago
Together AI

Together AI

We are seeking a software engineer to develop a Kubernetes-native control plane for provisioning and managing our GPU inference fleet. This role involves designing a manifest-driven API that allows the inference team to declare their needs, such as clusters, model deployments, or capacity adjustments. Our controllers will handle the underlying reconciliation, provider/runtime selection, and lifecycle management, abstracting away the complexities of the serving stack, scheduler, or hardware pool. You will also build systems to ensure fleet efficiency, including defragmentation, rebalancing logic for consolidating workloads, and scheduling/bin-packing improvements to maximize GPU utilization without compromising latency. The core objective is to decouple platform users from operational complexities while optimizing hardware capacity. This includes developing controllers, reconciliation loops, and self-service interfaces (API/CLI) for seamless operation, alongside event-driven systems for health monitoring, remediation, and utilization management, all designed to operate without human intervention. A product-oriented mindset is crucial, with a focus on building internal platforms and APIs that enhance the developer experience for consuming engineering teams. This role operates under a "you build it, you own it" philosophy, encompassing both software delivery and production support.

$160k - $280k

San Francisco onsite Full-time
KubernetesPythonGo +5 more

FAQs

Good questions, answered.

What to know about the AI and engineering roles listed here.

What counts as an AI Engineering job?

AiRagJobs lists engineering roles that build, deploy, or operate AI/ML systems — including LLM engineering, RAG (retrieval-augmented generation), AI agents, prompt engineering, AI infrastructure and MLOps, model serving/inference, and fine-tuning. It excludes non-engineering functions like AI sales or marketing roles.

What is the difference between an AI Engineer and a Machine Learning Engineer?

An AI Engineer typically builds applications on top of existing models — LLM integration, RAG pipelines, agent orchestration, and prompt design. A Machine Learning Engineer more often trains, fine-tunes, or productionizes custom models. Many companies use the titles interchangeably, so search both when browsing.

Does AiRagJobs include AI infrastructure and MLOps roles?

Yes. Titles such as AI Infrastructure Engineer, ML Platform Engineer, GPU Infrastructure Engineer, Inference Engineer, MLOps Engineer, and AI Site Reliability Engineer are all covered — these roles focus on the systems that serve and scale AI models rather than building models themselves.

Are remote AI and ML jobs available?

Yes. Filter by "Remote" on the jobs page to see fully remote AI/ML engineering roles, or browse the remote jobs feed directly.

What skills are most in demand for AI engineering roles?

The most commonly requested skills are LLM APIs (OpenAI, Anthropic, Gemini), RAG and vector databases (Pinecone, Weaviate, Qdrant), agent frameworks (LangGraph, CrewAI, AutoGen), inference/serving tools (vLLM, Ray Serve, TensorRT-LLM), and fine-tuning. Use the skill filters on the jobs page to browse by specific technology.

How fresh are the job listings?

Listings are sourced continuously from company career pages and refreshed automatically. Each job shows when it was posted, and the sitemap and API expose last-updated timestamps for every posting.

How much do AI Engineers get paid?

Compensation varies widely by role, seniority, and location. Where employers disclose a salary range, it is shown directly on the job listing — filter by salary range on the jobs page to narrow results to your target compensation.