Model Serving Jobs

AssemblyAI is a leading Voice AI company developing foundational speech models that power the next generation of voice applications. We operate at a significant scale, processing millions of hours of audio daily and serving billions of end-user experiences. This is a growth-stage opportunity within a capital-efficient company that offers significant ownership and impact. We foster a flat culture where employees are deeply involved in the details and can drive their work forward without bureaucratic hurdles. If you are passionate about Voice AI and want to solve interesting problems at scale, this is the place for you.

$270k - $310k

Harvey is transforming legal and professional services by combining frontier agentic AI with an enterprise-grade platform. As we scale rapidly and take on increasingly complex use cases, we are expanding our engineering team to accelerate our ability to ship fast, delight users, and build infrastructure that scales with us. As an Engineering Manager for AI Platform, you will lead and scale a team of engineers building the foundational systems that power AI across Harvey. This includes model access and routing, agent and workflow orchestration, evaluation frameworks, tool and context infrastructure, and the observability needed to run LLM-powered products reliably at scale. You will partner with Product, Research, and product engineering teams to turn fast-moving advances in applied AI into durable, reusable platform capabilities and set the technical direction for how teams across Harvey build, test, and ship AI features. This role requires both technical depth and product intuition, with a focus on making fast, high-quality decisions, working at the cutting edge of applied AI, and growing a team that embodies engineering excellence.

$272k - $355k

Abnormal AI is seeking a Senior Software Engineer to join the Detection Team, focused on building advanced technology to identify and stop email and cloud-based attacks. In this role, you will enhance the speed, responsiveness, stability, and confidence of feature development for the ML and Data Science teams. The ideal candidate possesses a first-principles approach to scalable, customer-centric solutions, a drive to solve meaningful problems, and an ownership-oriented outlook, with the ability to iterate quickly and autonomously on novel challenges.

$180k - $259k

Glean is seeking software engineers to contribute to building a leading search and assistant product for the workplace. Engineers will work across the technology stack, focusing on areas such as generative AI, Retrieval Augmented Generation (RAG), query and document understanding, domain-adapted language models, natural language question-answering, and evaluation. The role involves close interaction with customers to understand their challenges and applying the most effective tools to solve them.

$175k - $270k

Thinking Machines is building AI that extends human will and judgment. We are developing Tinker, a fine-tuning API that empowers researchers and developers to customize frontier AI models. This role focuses on building and enhancing our training systems for Tinker, specifically involving RL systems, numerics, and kernels, to enable users to train models with their own data and algorithms.

$350k - $475k

Deepgram is a leading platform for the Voice AI economy, offering real-time APIs for speech-to-text, text-to-speech, and building voice agents at scale. We empower over 1,300 organizations and 200,000 developers to create voice offerings powered by our advanced foundation models, accessible via cloud APIs or self-hosted solutions. Our AI-first approach means every team member is expected to actively use, experiment with, and build AI tools into their daily work, driving innovation and performance. We operate at the rapid pace of AI, requiring adaptability, continuous learning, and a proactive mindset to solve customer challenges and push technological boundaries.

$216k - $338k

NVIDIA is seeking a founding Director to establish and lead its Forward Deployed Engineering (FDE) organization for open models, specifically focusing on NVIDIA Nemotron and NVIDIA Cosmos across North America. The core mission is to transform open-source models into production-ready AI solutions tailored to specific domains, leveraging NVIDIA's partner ecosystem. This role involves setting the strategic direction for North America, building and scaling the FDE team, and fostering strong relationships with partners, customers, and NVIDIA leadership. The FDE team will work collaboratively with top partner engineering and research teams, co-leading joint customer engagements and developing best practices to ensure partner success and scalability. The ultimate goal is to empower organizations to build, own, and operate their AI on open-source models.

$320k - $489k

Blee is seeking an Applied Scientist to specialize in agent system design, focusing on architecting and implementing the next generation of intelligent compliance systems. This role involves designing frameworks where multiple agents, powered by custom and foundational models, collaborate to identify, track, and explain compliance flags. You will also be responsible for building and optimizing applied AI/ML pipelines, including model fine-tuning, prompt engineering, evaluation frameworks, and customer-specific AI configurations, and training models based on user feedback to deliver more targeted compliance recommendations.

$150k - $275k

SpaceXAI is building Super Intelligence to help humanity understand the universe, from supercomputers to large language models and user-facing applications. As a Software Engineer on the Model Lifecycle and Signals team, you will be responsible for integrating frontier models into user-facing products at a massive scale. This role involves deciding which models to use, monitoring their performance, and providing feedback to shape future model development. You will work at the intersection of infrastructure, data, and product, bringing models from preview to production, analyzing real-world data, and collaborating with research, inference, product, and data science teams to improve both products and models. Your contributions will have a compounding impact across all products and shipped models.

OpenAI

We are seeking a Machine Learning Engineer to develop and productionize new classification and evaluation capabilities. This role involves translating open-ended questions about model behavior into well-defined ML problems, building effective methods, and rigorously assessing their reliability. You will own classifiers throughout their lifecycle, from supervised fine-tuning and distillation to evaluation, deployment, and production inference. Collaborating closely with researchers to identify valuable problems and with software engineers to scale promising approaches will be key to success.

$381k - $510k

You.com is building the AI Search Infrastructure that powers modern AI systems, aiming to create a trusted knowledge layer for agents, applications, and enterprises. Our platform combines proprietary vertical indexes with LLM-optimized retrieval systems. We are seeking a Senior Backend Engineer to own and scale the inference gateway, which is the critical layer that handles all model calls. This role involves routing, fallback, caching, metering, and enforcement within strict latency budgets, sitting between model serving and product development.

$200k - $250k

Anthropic

Our Inference team is responsible for building and maintaining the critical systems that serve Claude to millions of users worldwide. We bring Claude to life by serving our models via the industry's largest compute-agnostic inference deployments. We are responsible for the entire stack from intelligent request routing to fleet-wide orchestration across diverse AI accelerators. The team has a dual mandate: maximizing compute efficiency to serve our explosive customer growth, while enabling breakthrough research by giving our scientists the high-performance inference infrastructure they need to develop next-generation models. We tackle complex, distributed systems challenges across multiple accelerator families and emerging AI hardware running in multiple cloud platforms.

€295k - €355k

NVIDIA is seeking a Senior Engineer to lead the performance modeling, evaluation, and post-training of large language models focused on code generation and agentic coding tasks. This role involves optimizing model capabilities, efficiency, and reliability by working at the intersection of model training, inference, and empirical analysis. You will design experiments to understand model behavior, identify performance bottlenecks, and develop post-training strategies to achieve measurable improvements in coding benchmarks and real-world software engineering tasks. Collaboration with research scientists, infrastructure engineers, and product teams will be key to translating model insights into production-ready features.

$224k - $431k

Databricks

The Foundation Model Inference team is the backbone of Databricks’ generative AI capabilities. We build the infrastructure that enables our customers to serve, scale, and optimize frontier models with enterprise-grade reliability and performance. Our Foundation Model APIs provide a unified platform that gives customers access to LLMs with the governance, flexibility, and scalability required for enterprise production workloads. We are looking for high-agency engineers who are excited to work on powering model inference at enterprise scale.

$190k - $265k

Databricks

Databricks' Model Serving product offers enterprises a unified, scalable, and governed platform for deploying and managing AI/ML models, including traditional ML and large language models. It provides real-time, low-latency inference, governance, monitoring, and lineage, making it a core pillar for customers operationalizing models at scale with strong SLAs and cost efficiency. As a Senior Engineer, you will be instrumental in shaping both the product experience and the foundational infrastructure of Model Serving. You will design and build high-throughput, low-latency inference systems for CPU and GPU workloads, influence architectural direction, and collaborate with cross-functional teams to deliver a world-class serving platform.

$166k - $225k

We are seeking a dynamic, mission-driven Developer Relations Manager to engage leading university-affiliated researchers and labs. In this role, you will work directly with researchers to accelerate the adoption of NVIDIA’s sophisticated AI and computing platforms. The ideal candidate combines deep technical expertise with strong experience in research engagement and a proven track record to translate advanced technologies into valuable, insight-focused outcomes.

$152k - $288k

NVIDIA is seeking a PhD intern to join the Deep Learning Efficiency Research (DLER) team, focusing on research that pushes boundaries and has real-world impact. The team's core areas include efficient diffusion language and multimodal generative models, as well as efficient agentic AI with hybrid inference orchestration. We are also interested in post-training model optimization, efficient architecture design, adaptive inference, and resource-efficient training and finetuning. You will collaborate with a research team that consistently publishes at top venues, contributing to technologies that can impact NVIDIA's products.

$0k - $0k

Physical Intelligence is building a platform to bring general-purpose AI into the physical world, powering robots with foundation models and learning algorithms. As an API Product Engineer, you will develop the core product surface that enables external companies to utilize Pi's models similarly to how they use LLM APIs today. This involves allowing them to bring their own data, fine-tune models, evaluate performance, and run low-latency inference within their own environments. You will own this entire experience, transforming close-knit collaborations into a scalable platform designed to support a vast number of robots.

Code Metal is seeking a Senior Software Engineer for its AI Platform team. This role involves designing, building, and operating the foundational AI enablement stack that powers the company's AI-driven code solutions. You will contribute to critical components such as GPU inference serving, model gateways, agent harnesses, context engineering, observability, and AI experimentation management. While this is an internal platform, it is built to product standards. The position offers the opportunity to eventually own a specific area, like model serving or agentic infrastructure, while also contributing broadly across the stack. The role requires a blend of strong software engineering skills with solid data science and AI research fundamentals, including occasional experimental work to inform platform decisions.

$170k - $210k

LlamaIndex is seeking a Head of Engineering to lead and develop their engineering team in building software that makes enterprise data useful for AI applications. This is a hands-on role where you will own engineering execution, technical direction, hiring, and team development. You will collaborate closely with company leadership and product teams to define product strategy and delivery. The ideal candidate has shipped and operated B2B API products, with experience in both model serving and model training, from experimentation and evaluation to deployment and production operations.

$200k - $300k