New: upload your resume and get matched to AI jobs. Free to start →

Find your next AI engineering role.

272 open roles at 309 companies, from top AI labs and YC startups. Refreshed every day.

Rather not search? Upload your resume and see the jobs you fit, best first. Get matched →

272 open positions

(Senior/Junior AI Inference Reliability Engineer, Compute Infra

4d ago
Together AI

Together AI

We are seeking a reliability-minded software engineer to manage the production operations of our Kubernetes-native control plane, which provisions and runs our GPU inference fleet. This platform team develops the manifest-driven API and controllers that enable the inference team to define their needs, such as cluster configurations, model deployments, and capacity adjustments, and ensures these are reconciled. Your role will be to maintain the reliability of this system in production, addressing issues like InfiniBand problems during node communication, diagnosing faults in hero-model and LoRA serving that generic health checks miss, and enhancing automated self-healing to permanently resolve recurring failure modes. The primary goal is to ensure the decoupling remains effective even during failures and to maintain fleet efficiency without introducing new instabilities. You will be responsible for inference serving SLOs and error budgets, leading incident response, and implementing systemic fixes based on postmortem analyses. Additionally, you will contribute to the reliability aspects of capacity efficiency, recognizing that misconfigured defragmentation or bin-packing can pose production risks. We value candidates with hands-on production reliability experience in large-scale distributed or GPU-serving systems, proficiency in troubleshooting InfiniBand/RDMA and Kubernetes controller/CRD-based platforms, and a software engineering approach to GPU infrastructure and its supporting tools.

$160k - $280k

San Francisco onsite Full-time
KubernetesPythonFine-Tuning +6 more

(Staff / Senior Staff) AI Inference Reliability Engineer, Compute Infra

11d ago
Together AI

Together AI

We are seeking a software engineer with a strong focus on reliability to manage the production operations of our Kubernetes-native control plane, which provisions and runs our GPU inference fleet. This platform team develops the API and controllers that enable the inference team to define their needs, such as clusters, model deployments, or capacity changes, and ensures these are reconciled. Your role will be to maintain this functionality in the real world by troubleshooting issues like InfiniBand problems, diagnosing serving faults, and enhancing our automated self-healing systems to permanently resolve recurring failures. The primary goal is to ensure the decoupling remains effective even during disruptions and to maintain fleet efficiency without introducing new instabilities. You will be responsible for inference serving SLOs and error budgets, leading incident response, and implementing lessons learned from postmortems into system changes. Additionally, you will address the reliability aspects of capacity efficiency, recognizing that misconfigured defragmentation or bin-packing can pose production risks. We value candidates with hands-on production reliability experience in large-scale distributed or GPU-serving systems, proficiency in troubleshooting InfiniBand/RDMA and Kubernetes controller-based platforms, and a software engineering approach to GPU infrastructure.

$250k - $320k

San Francisco onsite Full-time
KubernetesPythonFine-Tuning +6 more

FAQs

Good questions, answered.

What to know about the AI and engineering roles listed here.

What counts as an AI Engineering job?

AiRagJobs lists engineering roles that build, deploy, or operate AI/ML systems — including LLM engineering, RAG (retrieval-augmented generation), AI agents, prompt engineering, AI infrastructure and MLOps, model serving/inference, and fine-tuning. It excludes non-engineering functions like AI sales or marketing roles.

What is the difference between an AI Engineer and a Machine Learning Engineer?

An AI Engineer typically builds applications on top of existing models — LLM integration, RAG pipelines, agent orchestration, and prompt design. A Machine Learning Engineer more often trains, fine-tunes, or productionizes custom models. Many companies use the titles interchangeably, so search both when browsing.

Does AiRagJobs include AI infrastructure and MLOps roles?

Yes. Titles such as AI Infrastructure Engineer, ML Platform Engineer, GPU Infrastructure Engineer, Inference Engineer, MLOps Engineer, and AI Site Reliability Engineer are all covered — these roles focus on the systems that serve and scale AI models rather than building models themselves.

Are remote AI and ML jobs available?

Yes. Filter by "Remote" on the jobs page to see fully remote AI/ML engineering roles, or browse the remote jobs feed directly.

What skills are most in demand for AI engineering roles?

The most commonly requested skills are LLM APIs (OpenAI, Anthropic, Gemini), RAG and vector databases (Pinecone, Weaviate, Qdrant), agent frameworks (LangGraph, CrewAI, AutoGen), inference/serving tools (vLLM, Ray Serve, TensorRT-LLM), and fine-tuning. Use the skill filters on the jobs page to browse by specific technology.

How fresh are the job listings?

Listings are sourced continuously from company career pages and refreshed automatically. Each job shows when it was posted, and the sitemap and API expose last-updated timestamps for every posting.

How much do AI Engineers get paid?

Compensation varies widely by role, seniority, and location. Where employers disclose a salary range, it is shown directly on the job listing — filter by salary range on the jobs page to narrow results to your target compensation.