vLLM Jobs

Together AI

The Speculative Decoding team at Together AI focuses on developing draft models and algorithms to accelerate large language model generation without altering outputs. Their work integrates model training, inference algorithms, and serving systems, optimizing all three for reduced cost and latency. As a research intern, you will explore open questions in speculative decoding, including new drafter architectures, verification schemes, knowledge distillation, and adaptive drafters. You will train and fine-tune draft models at scale on GPU clusters and evaluate your solutions against realistic serving workloads, such as long-context and agentic traffic.

Prime Intellect is building the open superintelligence stack, a platform designed to unify compute, environments, evaluations, and deployment for frontier-scale AI development. We are creating open-source models trained for long-horizon tasks and the full-stack platform our research team uses to build them. We are looking for individuals passionate about building at the intersection of frontier research, real infrastructure, and go-to-market for a nascent category.

$150k - $350k

d-Matrix is seeking a Senior Staff Inference Performance Engineer to join their Inference Cloud Engineering team. This role is responsible for the end-to-end performance of workloads deployed on d-Matrix's inference services, ensuring they perform at least as well as on bare metal. The engineer will diagnose and isolate performance issues across the full serving path, from orchestration to inference internals, and will be responsible for either fixing problems directly or handing them off to the appropriate owning team with clear evidence. The role involves assessing scalability, defining node requirements for optimal deployment, and building telemetry and benchmarks to support these efforts. The ideal candidate will mentor other engineers in a profiling-first methodology.

$175k - $265k

NVIDIA Cloud Functions is seeking a motivated AI/ML Engineer with expertise in AI platform engineering at scale. The team develops and manages a serverless deployment platform designed to enable and scale AI inferencing workloads. This platform utilizes globally distributed orchestration of workloads on Kubernetes clusters backed by GPUs, operating in a cloud-agnostic manner. The role involves working with a team of innovative engineers to deliver a top-tier product for both external clients and internal NVIDIA teams, contributing to the forefront of cloud engineering paradigms for AI at scale.

$152k - $288k

You.com is building the AI Search Infrastructure that powers modern AI systems, aiming to create a trusted knowledge layer for agents, applications, and enterprises. Our platform combines proprietary vertical indexes with LLM-optimized retrieval systems. We are seeking a Senior Backend Engineer to own and scale the inference gateway, which is the critical layer that handles all model calls. This role involves routing, fallback, caching, metering, and enforcement within strict latency budgets, sitting between model serving and product development.

$200k - $250k

Code Metal is seeking a Senior Software Engineer for its AI Platform team. This role involves designing, building, and operating the foundational AI enablement stack that powers the company's AI-driven code solutions. You will contribute to critical components such as GPU inference serving, model gateways, agent harnesses, context engineering, observability, and AI experimentation management. While this is an internal platform, it is built to product standards. The position offers the opportunity to eventually own a specific area, like model serving or agentic infrastructure, while also contributing broadly across the stack. The role requires a blend of strong software engineering skills with solid data science and AI research fundamentals, including occasional experimental work to inform platform decisions.

$170k - $210k

Luma is seeking a Research Scientist/Engineer to build the systems that enable reinforcement learning (RL) at a frontier scale. This role involves coupling policy optimization with large fleets of inference workers, agentic environments, and reward/verification systems to translate model behavior into learning signals. RL is crucial for Luma's models to evolve from capable to useful. This is a full-loop systems problem encompassing training, rollout generation, environment execution, and reward computation across thousands of GPUs, requiring speed, stability, and correctness. The ideal candidate has experience with post-training LLMs using RL, building environments and verifiers, and debugging large-scale asynchronous rollout pipelines.

Luma is seeking a Software Engineer to own the serving of their AI models. This role involves integrating new model architectures into the inference engine, scaling deployments across thousands of machines, and optimizing the utilization of expensive GPU fleets while meeting internal service level objectives (SLOs). The work focuses on large-scale inference systems, including scheduling, fleet management, deployment pipelines, and ensuring reliability across diverse clusters and hardware providers. This position is ideal for a strong systems engineer comfortable with model serving and Kubernetes at scale, rather than those seeking pure modeling work.

Luma is seeking a Software Engineer to own the serving of their AI models. This role involves integrating new model architectures into the inference engine, scaling deployments across thousands of machines, and optimizing the utilization of expensive GPU fleets while meeting internal service level objectives (SLOs). The work focuses on large-scale inference systems, including scheduling, fleet management, deployment pipelines, and ensuring reliability across diverse clusters and hardware providers. This position is ideal for a strong systems engineer comfortable with model serving and Kubernetes at scale, rather than those seeking pure modeling work.

$195k - $345k

We are seeking a Senior Developer Relations Manager to build the developer ecosystem around NVIDIA Dynamo, an open-source framework for distributed AI inference. In this senior individual contributor role, you will help developers deploy and operate inference systems, build practical technical resources, and represent developer needs both in the community and inside NVIDIA. You will build the ecosystem with various inference projects through joint demos, integration guides, upstream contributions, and community programs. You will also develop runnable demos, sample code, technical documentation, and videos to explain Dynamo’s modular ecosystem and demonstrate its data center performance value. Additionally, you will represent the developer community inside NVIDIA by translating feedback into engineering and product requirements, influencing priorities, and communicating progress back to developers. You will advocate externally through technical talks, live demos, workshops, and office hours, helping developers troubleshoot real deployments and contribute to projects.

$184k - $357k

Code Metal is seeking a Staff AI Platform Engineer to architect and build the foundational AI enablement stack that internal engineering teams rely on. This role involves designing and operating production systems for GPU inference serving, model gateways, agent harnesses, context management, and observability. While primarily an engineering role focused on building and operating production systems, it also requires solid data science and AI research fundamentals to work closely with the AI Research team and conduct experiments when necessary. The platform is built to product standards, ensuring reliability and scalability for AI-driven code transpilation, modernization, optimization, modeling, and simulation solutions.

$205k - $250k

Sierra is seeking a Software Engineer for its Inference team to build the systems that power their AI agents. This role focuses on making foundation models fast, reliable, and efficient at scale. You will define the inference architecture, working on systems for serving, routing, capacity management, and optimization for latency, reliability, and cost. This is a systems-first role at the intersection of distributed infrastructure and AI, requiring strong distributed systems fundamentals and experience with complex systems problems.

$230k - $390k

We are a dedicated research lab focused on building, understanding, using, and risk-managing foundation models. Our mission is to advance AI research, cultivate the next generation of AI builders, and make significant contributions to a knowledge-driven economy. As part of our team, you will work on the core of cutting-edge foundation model training alongside world-class researchers, data scientists, and engineers. You will tackle fundamental and impactful challenges in AI development, contributing to groundbreaking AI solutions that have the potential to reshape industries. Strategic and innovative problem-solving skills are essential for establishing our institution as a global hub for high-performance computing in deep learning.

$150k - $450k

Percepta is transforming critical institutions with applied AI, embedding with industry-leading customers to drive AI transformation. We bring together forward-deployed expertise in engineering, product, and research, along with our in-house toolkit for rapidly deploying agentic workflows and strategic partnerships. As a Research Scientist - Reinforcement Learning, you will work at the intersection of RL research and real-world deployment, advancing decision-making capabilities for critical industries. You will collaborate closely with Embedded Product Managers and engineers to ensure our solutions transform how companies operate.

$200k - $400k

NVIDIA DGX Cloud provides AI services and endpoints for research and production workloads. We are seeking a Senior Production Engineer to develop software and automation that ensures these services are reliable, scalable, and safe to operate. The Production Engineering team focuses on large-scale distributed systems, including internal and external model endpoints, regional control plane services for workload orchestration and request routing, and the GPU/CPU compute infrastructure for inference and agentic workloads. Our work encompasses Kubernetes clusters across various cloud environments (AWS, Azure, Google Cloud, partner clouds) and on-premises deployments.

$184k - $357k

Runpod is seeking a Senior ML Systems Engineer focused on Inference to lead efforts in making LLM inference the fastest and most cost-efficient on their platform. This role involves owning LLM serving performance end-to-end, measuring, understanding, and improving it across various models, hardware, and workloads. The work directly impacts customer latency and cost, requiring a hands-on approach to identify and fix bottlenecks, and ensure reliable production deployment. The ideal candidate will be passionate about AI infrastructure and contribute to building the platform for the next generation of AI developers.

$150k - $220k

Ema builds AI Employees for HR, IT, and Finance, automating busy work across the employee experience from recruiting and onboarding to IT support and payroll. Our team, comprised of engineers from top tech companies and graduates from prestigious universities, is backed by leading investors and operates at the forefront of Agentic AI in production. We are developing systems where AI agents learn and improve from their real-world tasks, creating a continuous loop of production data, training, evaluation, and enhanced agent performance. The core challenge lies in determining the most effective interventions to improve agent behavior in live workflows.

As a Senior Engineering Manager, you will lead an engineering team focused on making large language models run faster, cheaper, and more reliably in production. This role balances team leadership with core systems and performance engineering on demanding models in production, requiring you to stay deeply technical and hands-on with the inference stack end to end. The work is applied, not academic, with optimizations landing in real customer deployments. You will tailor deployments to customer needs, take workloads from proof of concept to production, and ensure performance gains deliver clear business and customer value. This leadership role combines people management, technical direction, and hands-on coding and profiling, with a customer-facing component and elements of product and technical solutions leadership.

$250k - $300k

Together AI

Together AI is building the best inference infrastructure for voice applications, powering real-time voice agents and applications with best-in-class latency and reliability for speech-to-text and text-to-speech models. We are seeking a Senior ML Engineer to lead the model serving layer for voice workloads. You will optimize inference engines like TRT-LLM and SGLang to serve models such as Whisper, Parakeet, Orpheus, and Kokoro, focusing on pushing latency and throughput boundaries. This role involves profiling GPU utilization, designing batching strategies for streaming audio, and accelerating the transition of new model architectures from research to production. As a foundational hire on a small, high-impact team, you will address the unique challenges of voice inference, including streaming audio, real-time latency, and tokenization, shaping how Together serves voice models as the industry moves towards end-to-end speech-to-speech systems.

$200k - $260k

Together AI

Together AI's Model Shaping team focuses on tailoring open foundation models for downstream applications, building services that help ML developers select and improve models with domain-specific data. The team also develops efficient model training and evaluation methods, drawing from machine learning, NLP, and ML systems. As a Research Engineer, you will develop a platform for users to customize open-source models with their own data, working across training and inference stacks. You will build and enhance services for Fine-Tuning, Reinforcement Learning, and Evaluation, ensuring a smooth transition from post-training to production serving and optimizing inference engines for RL training. Collaboration with product, research, and engineering teams is key to maintaining API reliability and performance, ultimately contributing to the foundational layer of the open-source AI ecosystem.

$200k - $290k