Inference Jobs

Inworld is a research lab and inference provider focused on real-time AI for consumer-facing applications. We build first-party speech models, serve LLMs, and run the inference behind modular APIs designed for high-volume, real-time workloads. Our models and infrastructure support consumer applications across companions, healthcare, fitness, education, media, and more, serving over 10 trillion LLM tokens per month. We are looking for strong individuals from varied backgrounds who learn fast, thrive in ambiguity, and can demonstrate their experience in building, breaking, and understanding complex systems.

$250k - $300k

NVIDIA GPU Architecture Group is seeking a senior software engineer to automate and optimize performance analysis workflows for AI training and inference workloads. You will not only perform analysis but also reshape how it's done, building tools and workflows that scale across a broad audience of engineers. NVIDIA is a world-class leader in AI workload optimization, and your work will push these boundaries even further!

$184k - $357k

Scale AI

Scale's ML platform (RLXF) team builds our internal distributed framework for large language model training and inference. The platform powers MLEs, researchers, data scientists, and operators for fast and automatic training and evaluation of LLMs, as well as data quality evaluation. You will work closely across Scale’s ML teams and researchers to build the foundation platform that supports all our ML research and development, enabling the next generation of LLM training, inference, and data curation.

$190k - $237k

Relace is building a globally distributed inference cloud, competing on price, speed, and reliability. We process over 800 billion tokens daily and have experienced significant growth, backed by a $23 million Series A funding round. Our team is small, offering substantial responsibility and growth opportunities. We develop our own stack, running on Kubernetes across multiple clouds, where performance directly impacts business success. This role focuses on the lowest level of our stack, optimizing code that runs directly on GPUs to maximize work per second and dollar, shrink models for cost-efficiency without sacrificing quality, and eliminate GPU idle time.

SambaNova is seeking an Inference Systems Performance Architect to lead the end-to-end performance strategy for their AI inference platform. This role involves defining and driving technical direction in workload capture, benchmarking, modeling, and simulation. You will be instrumental in understanding how requests move through the system, from tokenization to fabric communication, and how deployments meet customer service level objectives. The work focuses on creating reproducible representations of real and agentic traffic, developing what-if capabilities for future configurations, and tackling complex challenges in heterogeneous inference environments.

$245k - $325k

SambaNova is seeking a Senior Software Engineer to join the ML Infrastructure team. This role is central to our mission as an inference-first company, focusing on designing, building, and operating the production-grade inference infrastructure that powers SambaNova's serving stack on our Reconfigurable Dataflow Unit (RDU) architecture. You will transform state-of-the-art inference techniques into reliable, high-throughput, low-latency services exposed to customers through SambaStack and SambaCloud. This position involves end-to-end ownership of systems including request scheduling, advanced decoding algorithms, caching layers, API surfaces, and accuracy infrastructure, and requires close collaboration with ML, compiler, runtime, and product teams.

$200k - $275k

Etched is building AI systems that accelerate its own development. As a Research Engineer on the RL Frontiers team, you will train proprietary frontier models to automate kernel development, ASIC engineering, production pipelines, and 3-D hardware operating systems. You will use proprietary simulators and environments to create custom data mixes and post-train models to SOTA performance on the company's hardest engineering tasks. The company is focused on inference and bets on transformer architectures, aiming to capture the entire inference market. Etched is a fully in-person team in San Jose, valuing engineering skills and expecting technical staff to contribute to both engineering and research across disciplines.

From $400k

Prime Intellect is building the open superintelligence stack, providing infrastructure for AI labs. Their platform, Lab, unifies compute, environments, evaluations, and training for post-training at scale, including SFT, RL, tool use, and agent workflows. They are focused on open-source models trained for long-horizon tasks and the platform used to build them. The company has raised significant funding from prominent investors and is seeking individuals to work at the intersection of frontier research, infrastructure, and go-to-market strategy for a nascent category.

$150k - $300k

NVIDIA is seeking a founding Director to establish and lead its Forward Deployed Engineering (FDE) organization for open models, specifically focusing on NVIDIA Nemotron and NVIDIA Cosmos across North America. The core mission is to transform open-source models into production-ready AI solutions tailored to specific domains, leveraging NVIDIA's partner ecosystem. This role involves setting the strategic direction for North America, building and scaling the FDE team, and fostering strong relationships with partners, customers, and NVIDIA leadership. The FDE team will work collaboratively with top partner engineering and research teams, co-leading joint customer engagements and developing best practices to ensure partner success and scalability. The ultimate goal is to empower organizations to build, own, and operate their AI on open-source models.

$320k - $489k

AI Labs at Salesforce is seeking deeply technical, product-savvy AI architects to evaluate AI technology for potential acquisitions and investments, and to accelerate portfolio companies post-acquisition. These leaders will provide technical depth during M&A diligence and help build common AI capabilities, models, and infrastructure that can be leveraged by acquired companies. The role involves assessing the technical viability of companies, identifying risks and opportunities, and advising on build-vs-buy decisions and the defensibility of underlying technology.

$197k - $365k

NVIDIA is seeking a Senior High Performance AI Engineer to develop groundbreaking multi-agent systems within the CUDA ecosystem. This role involves building innovative agentic runtimes and compiler-integrated orchestration that leverage NVIDIA's software stack to accelerate modern agent workloads powered by foundational models. You will contribute to new agent abstractions, GPU-centric runtimes, and system solutions that enhance agent planning, tool-use, and code generation. The position requires close collaboration with internal NVIDIA software and hardware teams to integrate cutting-edge developments into NVIDIA products.

$184k - $357k

NVIDIA is building GeForce G-Assist, an on-device AI assistant that integrates Small Language Models (SLMs), retrieval systems, and hybrid cloud capabilities for responsive, context-aware assistance within the GeForce ecosystem. This role involves close collaboration across engineering and product teams to ensure G-Assist performs reliably in real-world scenarios. The focus is on production model behavior rather than just benchmarks, with an emphasis on accuracy, robustness, and conversational reliability. The team works with SLM and VLM architectures, hybrid cloud/local model architectures, and optimizes local inference for performance.

$152k - $288k

Cohere

Cohere is seeking a Member of Technical Staff to join our Post-Training Data Infrastructure team. In this role, you will be responsible for building and managing the critical data pipelines that generate post-training data for our large language models (LLMs). This includes data for SFT, preference, and RL post-training, as well as synthetic and human-annotated data. You will collaborate closely with researchers and annotation teams to ensure rapid iteration on data, enabling them to keep pace with model development. Your work will be instrumental in advancing the capabilities of our foundation AI models and delivering value to enterprise customers.

CA$295k - CA$535k

NVIDIA is seeking a Senior Systems Software Engineer to join their Deep Learning Solutions team. This role focuses on optimizing deep learning inference for autonomous vehicles and robotics on edge devices. The ideal candidate will be a hands-on specialist capable of analyzing model architectures at the operator level, identifying performance bottlenecks through kernel trace analysis, and evaluating modern AI architectures on GPU and SOC platforms. This position directly contributes to enhancing real-time perception and response capabilities for autonomous systems, working at the intersection of model frameworks, compiler technology, and embedded hardware. Collaboration with automotive OEMs, robotics teams, and internal hardware groups is a key aspect of this role.

$224k - $357k

Reflection ai

Reflection is a research lab dedicated to making intelligence open and accessible. We build open models that empower users to control their intelligence and shape the future of AI. In this role, you will develop the critical evaluation infrastructure that researchers rely on to understand model capabilities, inform training decisions, and assess readiness for release. Your work will encompass evaluation authoring, distributed execution, and results analysis, with a specific emphasis on pre-training. You will collaborate closely with researchers to enhance the speed, reliability, and ease of use of evaluations, ensuring that the outcomes accurately reflect the models being assessed.

OpenRouter is seeking its first AI Inference SRE to ensure the operational health of its provider supply. This role will focus on making sure every endpoint is fast, correct, and available, and that problems are detected and routed around before customers are impacted. The SRE will be responsible for building and owning monitoring systems, improving detection and failover mechanisms, and managing incident response for provider-related issues. This position involves holding providers accountable through telemetry and SLO reporting, detecting quality regressions, automating manual operational tasks, and building tooling for capacity planning and launch readiness.

$245k - $295k

Anthropic

Our Inference team is responsible for building and maintaining the critical systems that serve Claude to millions of users worldwide. We bring Claude to life by serving our models via the industry's largest compute-agnostic inference deployments. We are responsible for the entire stack from intelligent request routing to fleet-wide orchestration across diverse AI accelerators. The team has a dual mandate: maximizing compute efficiency to serve our explosive customer growth, while enabling breakthrough research by giving our scientists the high-performance inference infrastructure they need to develop next-generation models. We tackle complex, distributed systems challenges across multiple accelerator families and emerging AI hardware running in multiple cloud platforms.

€235k - €295k

NVIDIA is seeking a Senior Engineer to lead the performance modeling, evaluation, and post-training of large language models focused on code generation and agentic coding tasks. This role involves optimizing model capabilities, efficiency, and reliability by working at the intersection of model training, inference, and empirical analysis. You will design experiments to understand model behavior, identify performance bottlenecks, and develop post-training strategies to achieve measurable improvements in coding benchmarks and real-world software engineering tasks. Collaboration with research scientists, infrastructure engineers, and product teams will be key to translating model insights into production-ready features.

$224k - $431k

Chai Discovery is building the computer-aided design suite for molecules, used by major pharmaceutical companies to power drug discovery programs. AI is revolutionizing life sciences, and our team is at the forefront of this transformation. We train frontier models that understand the fundamental principles of biochemical structure and interaction, coupled with a computational workspace that enables scientists to iterate on molecules from concept to medicine. This accelerates the development of novel therapeutics. This role focuses on building the AI platform that empowers scientists to direct research programs using agents. As Chai's models advance, scientists will increasingly focus on core scientific problems and hypotheses, delegating subtasks to intelligent agents. This requires creating innovative abstractions that facilitate seamless collaboration between people, agents, and Chai's foundation models.

NVIDIA is seeking a PhD intern to join the Deep Learning Efficiency Research (DLER) team, focusing on research that pushes boundaries and has real-world impact. The team's core areas include efficient diffusion language and multimodal generative models, as well as efficient agentic AI with hybrid inference orchestration. We are also interested in post-training model optimization, efficient architecture design, adaptive inference, and resource-efficient training and finetuning. You will collaborate with a research team that consistently publishes at top venues, contributing to technologies that can impact NVIDIA's products.

$0k - $0k