Model Serving Jobs

Moveworks is seeking a Senior Staff ML Engineer to join our AI Safeguards team. In this role, you will be instrumental in designing, building, and shipping critical safeguards within our agent harness. Your work will focus on protecting against prompt injection, single and multi-turn jailbreaks, toxicity, sensitive topics, and sensitive data exposure. You will develop robust evaluation frameworks to measure the efficacy of these safeguards, addressing both false positives and false negatives using diverse datasets. Collaboration with our Security team will be key, translating their findings into actionable eval cases and fixes, and co-developing mitigations. You will also select, integrate, and fine-tune detection models within a shared stack serving both commercial and government environments, while carefully balancing quality, latency, and cost for controls operating in the request path. Additionally, you will build configurable safeguard policies to empower customers in managing sensitive topics and monitor safeguard behavior in production, addressing any regressions.

CoreWeave is seeking a Junior Software Engineer to join their Inference team. In this role, you will be responsible for the end-to-end ownership of a production service, including feature development, maintenance of dashboards and alarms, and operational improvements. You will collaborate with cross-functional teams to enhance key service level indicators and contribute to the development of a cutting-edge inference platform that powers generative AI and real-time applications at scale.

$120k - $160k

CoreWeave is seeking a Software Engineer to join the Inference team, focusing on shipping production features that enhance model serving on their GPU platform. This role involves improving latency, reliability, and cost-efficiency. As an IC1, you will implement well-scoped changes, learn operational practices, and grow with mentorship from experienced engineers. CoreWeave is the essential cloud for AI, providing a platform of technology, tools, and teams that enable innovators to build and scale AI with confidence.

$92k - $135k

CoreWeave is seeking a Senior Engineer for its Benchmarking & Performance team, focusing on GPU kernel authoring and optimization. In this role, you will be responsible for writing, profiling, and tuning GPU kernels that are critical for large-scale model serving, aiming to maximize throughput and minimize latency. You will also contribute to industry-leading performance benchmarking publications. This position requires an individual who can lead designs, improve engineering standards, and deliver significant enhancements to latency, throughput, and reliability across the inference stack. You will collaborate with product, orchestration, and hardware teams to translate kernel-level optimizations into system-wide improvements and meet stringent Service Level Agreements (SLAs) at scale.

$182k - $242k

You'll join the Grok Voice Model team as a software engineer. You'll design, build, and run the software systems behind Grok Voice, which delivers natural, low-latency, multilingual spoken interactions across devices and in real time. The team owns the full engineering stack behind the model, from large-scale data processing to distributed training to production deployment.

$150k - $450k

Accenture

As a hands-on AI Infrastructure Architect, you will be responsible for designing and optimizing the AI and machine learning infrastructure that powers real-world applications. You will work alongside senior architects and engineers, applying your skills in coding, testing, configuring, deploying, monitoring, and troubleshooting AI systems and their underlying infrastructure. This role involves architecting and optimizing infrastructure components, writing and reviewing code and deployment scripts, designing and tuning compute resources like GPU clusters and distributed training environments, and deploying AI systems into production. You will also build and optimize data pipelines, monitor system health across InfraOps and MLOps, and troubleshoot complex issues. Additionally, you will mentor junior engineers, contribute to architectural decisions, and help establish best practices, making meaningful contributions to infrastructure that enables AI-driven business outcomes.

SpaceXSI is building Super Intelligence to help humanity understand the universe. Our teams are building the full Super Intelligence stack, from supercomputers to Grok models and a suite of applications. We're creating a powerful flywheel where more compute trains better models, better models power more useful products, and what we learn from people using those products shapes what we build next. The RL infrastructure team is looking for an engineer to help with low precision RL training and inference.

$180k - $440k

Together AI

We are seeking a software engineer with a strong focus on reliability to manage the production operations of our Kubernetes-native control plane, which provisions and runs our GPU inference fleet. This platform team develops the API and controllers that enable the inference team to define their needs, such as clusters, model deployments, or capacity changes, and ensures these are reconciled. Your role will be to maintain this functionality in the real world by troubleshooting issues like InfiniBand problems, diagnosing serving faults, and enhancing our automated self-healing systems to permanently resolve recurring failures. The primary goal is to ensure the decoupling remains effective even during disruptions and to maintain fleet efficiency without introducing new instabilities. You will be responsible for inference serving SLOs and error budgets, leading incident response, and implementing lessons learned from postmortems into system changes. Additionally, you will address the reliability aspects of capacity efficiency, recognizing that misconfigured defragmentation or bin-packing can pose production risks. We value candidates with hands-on production reliability experience in large-scale distributed or GPU-serving systems, proficiency in troubleshooting InfiniBand/RDMA and Kubernetes controller-based platforms, and a software engineering approach to GPU infrastructure.

$250k - $320k

Together AI

We are seeking a reliability-minded software engineer to manage the production operations of our Kubernetes-native control plane, which provisions and runs our GPU inference fleet. This platform team develops the manifest-driven API and controllers that enable the inference team to define their needs, such as cluster configurations, model deployments, and capacity adjustments, and ensures these are reconciled. Your role will be to maintain the reliability of this system in production, addressing issues like InfiniBand problems during node communication, diagnosing faults in hero-model and LoRA serving that generic health checks miss, and enhancing automated self-healing to permanently resolve recurring failure modes. The primary goal is to ensure the decoupling remains effective even during failures and to maintain fleet efficiency without introducing new instabilities. You will be responsible for inference serving SLOs and error budgets, leading incident response, and implementing systemic fixes based on postmortem analyses. Additionally, you will contribute to the reliability aspects of capacity efficiency, recognizing that misconfigured defragmentation or bin-packing can pose production risks. We value candidates with hands-on production reliability experience in large-scale distributed or GPU-serving systems, proficiency in troubleshooting InfiniBand/RDMA and Kubernetes controller/CRD-based platforms, and a software engineering approach to GPU infrastructure and its supporting tools.

$160k - $280k

Figma is seeking a hands-on engineering leader to manage and grow its AI Infrastructure team. This team builds the foundational inference systems that power Figma's AI products, ensuring reliability, observability, and seamless integration of cutting-edge models. The role involves defining and executing technical strategy, collaborating across engineering, and directly contributing to the core AI platform. This is an opportunity to join a rapidly growing team at the ground floor and shape the future of AI at Figma.

$258k - $376k

SpaceXSI is building the high-performance inference platform that serves Grok to millions of users daily with exceptional speed and reliability. As a Member of Technical Staff - Inference, you will be instrumental in designing and optimizing large-scale model serving systems from the ground up. This includes owning distributed infrastructure components like global KV cache, continuous batching, load balancing, and auto-scaling, as well as delving into deep low-level optimizations such as GPU kernels, quantization, and speculative decoding. This is a high-impact role where your contributions will directly influence the speed and reliability of user interactions with Grok at a massive scale.

$180k - $440k

Nebius is building a full-stack AI cloud platform for the global AI economy, supporting developers and enterprises from data and model training through to production deployment. As a Forward Deployed Engineer, you will work alongside Physical AI customers to understand their workloads and production requirements, translating these into actionable insights for our core infrastructure products. You will also investigate and prototype new product opportunities based on unmet customer needs, helping to shape the future of Nebius' offerings. This hands-on role involves writing code, deploying and benchmarking models, building agents, and troubleshooting infrastructure in close collaboration with customers, NVIDIA, Product, and Engineering teams.

Nebius is building a full-stack AI cloud platform to support developers and enterprises in AI/ML infrastructure, from data and model training to production deployment. This role focuses on the infrastructure for large-scale AI training and model post-training capabilities, specifically for frontier model improvement. The work involves distributed systems, GPU performance, model training frameworks, RL pipelines, and production engineering. The Senior Machine Learning Engineer will own ML work end-to-end, translating ambiguous goals into concrete experiments, implementing and debugging training and RL recipes, building supporting systems, and delivering measurable improvements in model quality, throughput, and reliability. This is a hands-on role requiring independent debugging of both model behavior and distributed training failures.

$195k - $262k

Nebius is building a full-stack AI cloud platform for the global AI economy, supporting developers and enterprises from data and model training through to production deployment. As a Forward Deployed Engineer, you will work directly with Physical AI customers to understand their workloads and production requirements. Your insights will shape our core infrastructure products and inform future development by connecting needs across Physical AI industries with product decisions. You will also investigate opportunities for new Nebius products by developing hypotheses about unmet needs, examining demand, and building prototypes to test with customers. This hands-on role involves writing code, deploying and benchmarking models, building agents, and troubleshooting infrastructure alongside customer engineers, with close collaboration across customers, NVIDIA, Product, and Engineering.

$180k - $224k

Enterprise AI is building the platform that enables reliable deployment of AI systems across enterprise environments. As a Platform Engineer, you will build and operate the core of Turing’s Enterprise AI platform, including platform services, the model gateway, multi-tenant infrastructure, CI/CD, infrastructure as code, and observability. The central challenge is creating repeatable deployments across environments Turing may not control, including client cloud accounts, dedicated instances, and sovereign installations. This role requires hands-on software engineering and operational ownership, with a focus on building production software and contributing deeply to platform services.

You will design and build the core infrastructure that powers AI inference across Cloudflare's global network. This includes real-time voice, frontier open LLMs, and customer-deployed models running on a heterogeneous fleet of GPUs and next-generation accelerators. You will work alongside AI/ML engineers, hardware partners, and Cloudflare product teams to solve complex problems in distributed systems and high-performance computing. The role involves addressing challenges such as sub-second model cold starts, multi-accelerator workload scheduling, efficient KV cache management, and developing a model deployment platform for both internal use and customer models. This is an opportunity to be at the center of building a novel AI inference platform embedded in the fabric of the internet, defining how AI runs at the edge of the network.

$194k - $266k

You will help define how machine learning models run across Cloudflare’s global network, from frontier open LLMs and real-time voice models to customer-deployed models served on heterogeneous GPUs and next-generation accelerators. You’ll work with systems engineers, product teams, hardware partners, and AI/ML engineers to bring models into production with low latency, strong reliability, and efficient resource use. This role combines applied ML, inference optimization, evaluation, and production engineering, with a focus on benchmarking models, improving serving performance, validating quality, and building tooling that helps Cloudflare and its customers ship AI applications at Internet scale.

$194k - $266k

Inworld is a research lab and inference provider focused on real-time AI for consumer-facing applications. We build first-party speech models, serve LLMs, and run the inference behind modular APIs designed for high-volume, real-time workloads. Our models and infrastructure support consumer applications across companions, healthcare, fitness, education, media, and more, serving over 10 trillion LLM tokens per month. We are looking for strong individuals from varied backgrounds who learn fast, thrive in ambiguity, and can demonstrate their experience in building, breaking, and understanding complex systems.

$250k - $300k

Scale AI

Scale is seeking a Staff Machine Learning Engineer for its Public Sector team to lead the design and deployment of agentic AI systems in critical government environments. This role involves working at the intersection of agentic ML, systems engineering, and applied research to build foundational infrastructure for AI systems that reason, plan, and act reliably at a national scale. You will partner directly with U.S. defense and intelligence agencies, contributing to flagship programs that advance agentic AI for geospatial reasoning, planning, and decision support. As a Staff ML Engineer, you will set technical direction, own core architectures, and translate ambitious ideas into production systems trusted by government operators.

$274k - $343k

Scale AI

Scale's ML platform (RLXF) team builds our internal distributed framework for large language model training and inference. The platform powers MLEs, researchers, data scientists, and operators for fast and automatic training and evaluation of LLMs, as well as data quality evaluation. You will work closely across Scale’s ML teams and researchers to build the foundation platform that supports all our ML research and development, enabling the next generation of LLM training, inference, and data curation.

$190k - $237k