Model Serving Jobs

Together AI

Together AI is building the best inference infrastructure for voice applications, powering real-time voice agents and applications with best-in-class latency and reliability for speech-to-text and text-to-speech models. We are seeking a Senior ML Engineer to lead the model serving layer for voice workloads. You will optimize inference engines like TRT-LLM and SGLang to serve models such as Whisper, Parakeet, Orpheus, and Kokoro, focusing on pushing latency and throughput boundaries. This role involves profiling GPU utilization, designing batching strategies for streaming audio, and accelerating the transition of new model architectures from research to production. As a foundational hire on a small, high-impact team, you will address the unique challenges of voice inference, including streaming audio, real-time latency, and tokenization, shaping how Together serves voice models as the industry moves towards end-to-end speech-to-speech systems.

$200k - $260k

Together AI

Together AI is seeking a Machine Learning Engineer to join our Inference Engine team, focusing on optimizing and enhancing the performance of our AI inference systems. This role involves working with state-of-the-art large language models and ensuring they run efficiently and effectively at scale. You will have the chance to collaborate closely with AI researchers and engineers to create cutting-edge AI solutions and shape the future of AI inference.

$200k - $300k

Together AI

Together AI is building the best inference infrastructure for voice applications, powering production-grade, real-time voice agents and applications with best-in-class latency and reliability for speech-to-text and text-to-speech models. We are seeking a Staff ML Engineer to lead the model serving layer for voice workloads. This role involves hands-on optimization of inference engines and models, profiling GPU utilization, designing batching strategies for streaming audio, and ensuring rapid deployment of new model architectures from research to production. This is a foundational hire on a small, high-impact team, addressing the unique challenges of voice inference such as streaming audio, tokenization, and real-time latency budgets, and shaping the future of voice model serving.

$220k - $280k

Databricks

As a Senior Applied ML Engineer on the Applied AI team, you will leverage machine learning, scheduling, and optimization algorithms to enhance the efficiency and performance of Databricks' infrastructure. Your responsibilities will encompass the full technology stack, from cluster management to query compilation, tackling complex, high-impact engineering challenges to deliver optimized and cost-effective workloads for customers. This role is crucial for accelerating serverless growth, building end-to-end ML4Sys solutions, shaping applied ML strategy, and driving the deployment of state-of-the-art models.

$166k - $210k

Databricks

At Databricks, we are passionate about enabling data and AI teams to solve the world's toughest problems. We build and run the world's best data and AI infrastructure platform. As part of the AI Platform team, you'll build the substrate that powers everything from data apps, AI agents, model training, model serving, and vector search. You'll be joining a high-agency, high-visibility team operating at the frontier of AI infrastructure with deep ties to research, product, and real-world enterprise use cases. Databricks Mosaic AI is one of our fastest-growing businesses helping thousands of our customers democratize AI within their organizations. We’re building the infrastructure that powers the next generation of AI.

$165k - $220k

Databricks

Databricks is seeking a Staff Backline Engineer to join their critical bridge team between Frontline Support and Engineering. This role focuses on handling complex technical issues and escalations within the Data and AI ecosystem, with a strong emphasis on customer success and proactive issue resolution. The engineer will be a technical expert and escalation point for challenging ML/AI issues, working collaboratively with Support, Engineering, Product, and customers to troubleshoot, identify root causes, and drive resolutions. The team prioritizes automation and tooling to enhance troubleshooting efficiency and improve platform supportability.

$0k - $0k

Databricks

As a staff software engineer for GenAI inference, you will lead the architecture, development, and optimization of the inference engine that powers Databricks Foundation Model API. You’ll bridge research advances and production demands, ensuring high throughput, low latency, and robust scaling. Your work will encompass the full GenAI inference stack: kernels, runtimes, orchestration, memory, and integration with frameworks and orchestration systems.

$191k - $233k

We are seeking a mission-driven Developer Relations Manager focused on Foundational AI Research to engage leading academic labs advancing the next generation of AI models, systems, and methods. In this role, you will work directly with top researchers building frontier AI systems, including large language models, multimodal models, reasoning systems, training methods, inference systems, model serving, and scalable AI infrastructure. You will help researchers adopt NVIDIA’s AI and accelerated computing platforms to push the boundaries of model performance, efficiency, and scale.

$152k - $288k

Modal is building the next-generation infrastructure layer for AI, enabling companies to serve low-latency inference, fine-tune models, and access production-ready sandboxes at scale. Our platform provides instant GPU access, sub-second container starts, and native storage, supporting category-defining customers. We are seeking a leader to own the direction and execution of our LLM inference platform, a critical component of our offering that delivers frontier performance with best-in-class elasticity and developer experience. This is a hands-on leadership role where you will split your time between technical contributions, product shaping, and people management, setting direction, removing blockers, and fostering a strong engineering culture.

$300k - $350k

Anthropic

The Encodings Infra team at Anthropic maintains the libraries that engineers and researchers use to encode text and multimodal data for Claude. This team also defines Claude's prompt structure, including how user turns are represented and how Claude interacts with tools. As a Software Engineer on this team, you will be responsible for the design and upkeep of these libraries, ensuring their APIs are intuitive, performance is optimized, and abstractions are robust. Your work will span across various systems, from pretraining to finetuning and the API, requiring close collaboration with researchers and engineers to facilitate the rapid transition of new encoding ideas from experimentation to production.

$320k - $405k

NVIDIA is seeking a Principal Systems Software Engineer for Semiconductor Inspection to join their team in Santa Clara. This role involves defining, building, and evaluating AI systems for semiconductor manufacturing, focusing on inspection and metrology. The work spans computer vision, time-series modeling, multimodal AI, and anomaly detection, with a strong emphasis on adapting and deploying these technologies within partner environments. The goal is to create integrated, reliable, and fast AI solutions that can be operated by fabs and equipment vendors, even in air-gapped facilities with limited data and tight production budgets.

$272k - $431k

We are building the AI layer of the Radio Access Network, applying agentic AI, machine learning, and LLMs to real RAN challenges. This role is for a Technical Lead to own the end-to-end architecture and technical direction of these AI systems. It is a hands-on senior individual contributor role, combining architecture, coding, technical leadership, and mentorship. You will work alongside RAN, PHY, and software teams, building production-grade AI solutions for real cellular networks.

CHAOS Industries is seeking a mission-oriented Applied AI Engineer to develop, integrate, and deploy AI/ML capabilities across its defense product lines. This role involves working with data scientists, software engineers, and product teams to conduct advanced AI research and translate it into reliable, real-world software, particularly for defense applications requiring extreme accuracy under challenging conditions. The ideal candidate possesses strong data science and software engineering fundamentals, a track record of technical excellence, and experience applying AI/ML to real products, with a preference for backgrounds in aerospace, defense, or similar mission-driven environments. Comfort with independent operation, rapid learning of new technical domains, cross-team collaboration, and moving prototypes to production with minimal oversight is essential.

$150k - $170k

Plaid is seeking a Senior Research Scientist for its Fraud Data team. This role focuses on applied research to develop next-generation fraud detection models using advanced techniques like Graph Neural Networks and Transformer-based foundation models. You will partner closely with Machine Learning Engineers and Data Scientists to uncover new signals, explore innovative modeling approaches, and translate promising research into production-ready solutions that strengthen fraud detection across Plaid’s network. The team owns the full model lifecycle, from data processing and experimentation to feature pipelines, model serving, and monitoring. This position offers the opportunity to work with large-scale financial data and contribute to building tools that millions of people rely on for their financial health.

$229k - $315k

NVIDIA is seeking a Senior Software Engineer to specialize in Deep Learning Inference. This role involves designing, building, and optimizing GPU-accelerated software for AI applications, focusing on high-performance deep learning frameworks like SGLang and vLLM for efficient large-scale model serving and inference. You will contribute to improving these platforms, enabling the smooth deployment of advanced language models and working with the deep learning community to implement and release the latest algorithms. The work will concentrate on enhancing the performance of state-of-the-art LLM and Generative AI models across various NVIDIA accelerators, utilizing open-source tools and kernels for optimization.

$184k - $357k

Anthropic

The Safeguards ML Inference Path team designs, builds, and operates the production infrastructure that powers Claude's ML-based safety systems. This role involves collaborating closely with safety researchers and inference engineers to bring new classifiers and novel classes of ML defenses to production, owning the research-to-production transfer of new safety technologies critical for every Claude model launch. The team builds for scale, serving thousands of ML classifiers for all requests on the token generation path and for every platform Claude runs on. Engineers will work at the intersection of machine learning, large-scale distributed systems, and AI safety, developing platforms and tools that enable safeguards to operate reliably at scale, with tooling and infrastructure used for every model launch.

$320k - $485k

Fireworks is seeking new graduates for its Member of Technical Staff role, offering a unique opportunity to work on real AI systems in production from day one. As an MTS, you will write and ship code for one of the world's busiest inference platforms, serving hundreds of models at massive scale. The company matches new hires to teams based on their strengths and interests, with potential work spanning inference, model training, distributed systems, cloud infrastructure, developer platforms, or full-stack application engineering. You will be paired with a senior engineer mentor and undergo a structured ramp-up, starting with well-scoped problems and progressing to owning features and systems independently. Fireworks emphasizes a fast-paced environment with a high bar and a commitment to employee development.

$160k - $180k

Fireworks is seeking PhD graduates in Computer Science, Computer Engineering, or Electrical Engineering to join their Systems Infrastructure team. This role is for individuals who want to apply their research ideas to real-world, large-scale AI infrastructure. You will be responsible for designing and building the core systems that power Fireworks' AI platform, including schedulers, storage systems, and networks, ensuring efficient utilization of hardware and low inference latency. You will work on challenging problems related to compute placement, efficient data movement, and high-performance networking for AI workloads, with the opportunity to see your work ship in a production environment.

$200k - $230k

Blee is an AI-first marketing content compliance platform trusted by major companies like PayPal, NerdWallet, and Brex. We are a YC-backed startup defining a new category in compliance across various industries. We value in-person collaboration but offer remote opportunities for exceptional candidates, with regular off-sites and opportunities to connect across offices. You will join a team focused on clarity, ownership, and building impactful solutions.

$150k - $275k

Decagon is seeking a Research Engineer focused on Audio and Speech to build and productionize models and agent harnesses for real-time voice agents. This role involves advancing multimodal and full-duplex systems capable of natural, real-time listening, reasoning, speaking, and responding. The ideal candidate will own their work end-to-end, ship impactful improvements, and make high-level technical decisions in a collaborative environment.

$200k - $400k