Inference Jobs

As a Senior Engineering Manager, you will lead an engineering team focused on making large language models run faster, cheaper, and more reliably in production. This role balances team leadership with core systems and performance engineering on demanding models in production, requiring you to stay deeply technical and hands-on with the inference stack end to end. The work is applied, not academic, with optimizations landing in real customer deployments. You will tailor deployments to customer needs, take workloads from proof of concept to production, and ensure performance gains deliver clear business and customer value. This leadership role combines people management, technical direction, and hands-on coding and profiling, with a customer-facing component and elements of product and technical solutions leadership.

$250k - $300k

Together AI

Together AI is building the best inference infrastructure for voice applications, powering production-grade, real-time voice agents and applications with best-in-class latency and reliability for speech-to-text and text-to-speech models. We are seeking a Staff ML Engineer to lead the model serving layer for voice workloads. This role involves hands-on optimization of inference engines and models, profiling GPU utilization, designing batching strategies for streaming audio, and ensuring rapid deployment of new model architectures from research to production. This is a foundational hire on a small, high-impact team, addressing the unique challenges of voice inference such as streaming audio, tokenization, and real-time latency budgets, and shaping the future of voice model serving.

$220k - $280k

Together AI

Together AI is building the best inference infrastructure for voice applications, powering real-time voice agents and applications with best-in-class latency and reliability for speech-to-text and text-to-speech models. We are seeking a Senior ML Engineer to lead the model serving layer for voice workloads. You will optimize inference engines like TRT-LLM and SGLang to serve models such as Whisper, Parakeet, Orpheus, and Kokoro, focusing on pushing latency and throughput boundaries. This role involves profiling GPU utilization, designing batching strategies for streaming audio, and accelerating the transition of new model architectures from research to production. As a foundational hire on a small, high-impact team, you will address the unique challenges of voice inference, including streaming audio, real-time latency, and tokenization, shaping how Together serves voice models as the industry moves towards end-to-end speech-to-speech systems.

$200k - $260k

Together AI

Together AI is seeking a Machine Learning Engineer to join our Inference Engine team, focusing on optimizing and enhancing the performance of our AI inference systems. This role involves working with state-of-the-art large language models and ensuring they run efficiently and effectively at scale. You will have the chance to collaborate closely with AI researchers and engineers to create cutting-edge AI solutions and shape the future of AI inference.

$200k - $300k

Together AI

Together AI's Model Shaping team focuses on tailoring open foundation models for downstream applications, building services that help ML developers select and improve models with domain-specific data. The team also develops efficient model training and evaluation methods, drawing from machine learning, NLP, and ML systems. As a Research Engineer, you will develop a platform for users to customize open-source models with their own data, working across training and inference stacks. You will build and enhance services for Fine-Tuning, Reinforcement Learning, and Evaluation, ensuring a smooth transition from post-training to production serving and optimizing inference engines for RL training. Collaboration with product, research, and engineering teams is key to maintaining API reliability and performance, ultimately contributing to the foundational layer of the open-source AI ecosystem.

$200k - $290k

Crusoe is seeking a Senior Staff Applied AI Inference Engineer to accelerate the abundance of energy and intelligence by optimizing AI infrastructure. This role focuses on making large language models run faster, cheaper, and more reliably in production by owning the inference stack end-to-end. You will be responsible for profiling performance, implementing modern optimization techniques, and diving deep into serving code. The work is applied, directly impacting customer deployments, and involves tailoring solutions to specific models, traffic patterns, latency targets, and cost constraints. You will collaborate with customer engineering teams, guide workloads from proof of concept to production, and ensure engineered gains are realized by users. This is a hands-on engineering role requiring coding, profiling, and low-level optimization, with a customer-facing component and elements of product and technical solutions work.

$250k - $300k

AssemblyAI is seeking a Software Engineer to bridge the gap between cutting-edge AI research and production-ready products. You will join a dynamic engineering team responsible for transforming new model capabilities from initial concepts into robust, well-documented APIs that handle production traffic at scale. This role involves designing and launching customer-facing APIs, scaling inference infrastructure, and collaborating with researchers and customers to ensure our products meet user needs and drive future development.

$190k - $225k

Databricks

As a staff software engineer for GenAI inference, you will lead the architecture, development, and optimization of the inference engine that powers Databricks Foundation Model API. You’ll bridge research advances and production demands, ensuring high throughput, low latency, and robust scaling. Your work will encompass the full GenAI inference stack: kernels, runtimes, orchestration, memory, and integration with frameworks and orchestration systems.

$191k - $233k

Databricks

Databricks AI is seeking a Principal AI Research Scientist to lead a team focused on advancing the state of the art in large-scale machine learning, particularly in LLM training and inference efficiency. This role involves defining and executing a research roadmap to improve how customers train, serve, and adapt LLMs at scale, working closely with product, data, and engineering leaders. The ideal candidate will drive algorithmic innovations, optimize ML systems for distributed training and inference, and translate research breakthroughs into customer-impacting capabilities within the Databricks AI platform. This position also requires fostering a culture of scientific excellence, representing Databricks AI externally, and mentoring research talent.

$270k - $340k

NVIDIA is seeking a Senior Integration Engineer to join their autonomous driving team. This role focuses on accelerating the development, integration, evaluation, and deployment of end-to-end driving models. You will collaborate across model development, data, simulation, systems software, and vehicle engineering teams to transform evolving AI models into reliable autonomous driving functionality on NVIDIA's computing platforms. The position involves working with large-scale training infrastructure, simulation environments, and production vehicle platforms.

$152k - $288k

We are seeking a Staff/Senior Software Engineer to join our Agent Engineering team, focusing on the Helios project. This role involves building and strengthening the foundations of our agent runtime, enabling the creation of new agent-driven products. You will develop systems that allow agents to manage long-running work from assignment to completion, ensuring agent activity is visible, steerable, and connected to relevant projects, people, evidence, deadlines, and deliverables. Additionally, you will construct the framework for agents to safely interact with internal systems and external services, and connect agents to our retrieval and knowledge systems to ground their reasoning, memory, citations, and generated artifacts in authoritative evidence. A key part of this role is improving the evaluation and release discipline for agent behavior, traces, and simulations to enhance quality while controlling latency, cost, and risk.

Modal is building the next-generation infrastructure layer for AI, enabling companies to serve low-latency inference, fine-tune models, and access production-ready sandboxes at scale. Our platform provides instant GPU access, sub-second container starts, and native storage, supporting category-defining customers. We are seeking a leader to own the direction and execution of our LLM inference platform, a critical component of our offering that delivers frontier performance with best-in-class elasticity and developer experience. This is a hands-on leadership role where you will split your time between technical contributions, product shaping, and people management, setting direction, removing blockers, and fostering a strong engineering culture.

$300k - $350k

OpenAI

We are seeking an AI Systems Engineer to join our pAGI Infra team, which builds and operates the systems essential for large-scale model training and evaluation. This role involves owning projects from identifying bottlenecks and designing solutions to deployment and operation, focusing on distributed systems engineering, performance optimization, and collaboration with researchers. You will directly contribute to improving the speed and reliability of research by potentially building shared grading services, optimizing resource allocation, or integrating new training stacks into production.

$266k - $500k

Ambient.ai is seeking an AI Infrastructure Software Engineer to design, build, and optimize the AI infrastructure powering their real-time intelligence platform. This role involves working on systems to run state-of-the-art deep learning models, including computer vision, large language models (LLMs), large vision models (LVMs), and multimodal AI systems, across vast amounts of video data in real time. You will focus on scaling infrastructure for inference, evaluation, and continuous model improvement, partnering closely with research scientists and product engineering teams to bring AI advancements into production.

$168k - $205k

NVIDIA is seeking a highly technical Engagement Lead to drive the integration of Generative and Agentic AI software with enterprise ISV partners. In this role, you will serve as a trusted advisor to Software Platform Architects and unite teams behind a unified technical vision. You will remain hands-on, designing and shipping production agentic systems for demanding workloads while accelerating the adoption of NVIDIA's stack across key business initiatives. This position requires versatility at the intersection of platform strategy, a strong AI/ML background with hands-on execution, and the communication skills to engage with senior leaders. It's a high-impact role offering broad skill development and exposure to the cutting edge of Agentic AI, with insights directly shaping NVIDIA's product development.

$224k - $431k

NVIDIA is seeking a Senior Software Engineer to specialize in Deep Learning Inference. This role involves designing, building, and optimizing GPU-accelerated software for AI applications, focusing on high-performance deep learning frameworks like SGLang and vLLM for efficient large-scale model serving and inference. You will contribute to improving these platforms, enabling the smooth deployment of advanced language models and working with the deep learning community to implement and release the latest algorithms. The work will concentrate on enhancing the performance of state-of-the-art LLM and Generative AI models across various NVIDIA accelerators, utilizing open-source tools and kernels for optimization.

$184k - $357k

Anthropic

The Safeguards ML Inference Path team designs, builds, and operates the production infrastructure that powers Claude's ML-based safety systems. This role involves collaborating closely with safety researchers and inference engineers to bring new classifiers and novel classes of ML defenses to production, owning the research-to-production transfer of new safety technologies critical for every Claude model launch. The team builds for scale, serving thousands of ML classifiers for all requests on the token generation path and for every platform Claude runs on. Engineers will work at the intersection of machine learning, large-scale distributed systems, and AI safety, developing platforms and tools that enable safeguards to operate reliably at scale, with tooling and infrastructure used for every model launch.

$320k - $485k

Crusoe is seeking a Staff Applied AI Inference Engineer to accelerate the abundance of energy and intelligence by optimizing large language models for production environments. This role involves owning the inference stack end-to-end, focusing on profiling, applying modern optimization techniques, and deep dives into serving code to enhance speed, reduce cost, and improve reliability. The work is hands-on, involving coding, profiling, and low-level optimization, with a customer-facing component to tailor deployments to specific needs and ensure performance gains are realized in real-world scenarios. You will transition workloads from proof-of-concept to fully monitored production services, making this a core systems and performance engineering role on demanding AI workloads.

$215k - $260k

Decagon is seeking a Research Engineer focused on Audio and Speech to build and productionize models and agent harnesses for real-time voice agents. This role involves advancing multimodal and full-duplex systems capable of natural, real-time listening, reasoning, speaking, and responding. The ideal candidate will own their work end-to-end, ship impactful improvements, and make high-level technical decisions in a collaborative environment.

$200k - $400k

Mercor is seeking a Software Engineer for their Applied AI - Frontier Engineering team to build, deploy, and operate systems that bridge frontier AI research and data delivery. This role involves high ownership and deep technical work, tackling ill-defined problems, prototyping with researchers and customers, and scaling systems from experiments to production. You will manage projects end-to-end, including requirements gathering, data creation pipelines, and improving model-adjacent infrastructure, while collaborating with AI labs and internal teams to deliver impactful applied AI solutions.

$160k - $500k