SGLang Jobs

You.com is building the AI Search Infrastructure that powers modern AI systems, aiming to create a trusted knowledge layer for agents, applications, and enterprises. Our platform combines proprietary vertical indexes with LLM-optimized retrieval systems. We are seeking a Senior Backend Engineer to own and scale the inference gateway, which is the critical layer that handles all model calls. This role involves routing, fallback, caching, metering, and enforcement within strict latency budgets, sitting between model serving and product development.

$200k - $250k

Luma is seeking a Software Engineer to own the serving of their AI models. This role involves integrating new model architectures into the inference engine, scaling deployments across thousands of machines, and optimizing the utilization of expensive GPU fleets while meeting internal service level objectives (SLOs). The work focuses on large-scale inference systems, including scheduling, fleet management, deployment pipelines, and ensuring reliability across diverse clusters and hardware providers. This position is ideal for a strong systems engineer comfortable with model serving and Kubernetes at scale, rather than those seeking pure modeling work.

Luma is seeking a Software Engineer to own the serving of their AI models. This role involves integrating new model architectures into the inference engine, scaling deployments across thousands of machines, and optimizing the utilization of expensive GPU fleets while meeting internal service level objectives (SLOs). The work focuses on large-scale inference systems, including scheduling, fleet management, deployment pipelines, and ensuring reliability across diverse clusters and hardware providers. This position is ideal for a strong systems engineer comfortable with model serving and Kubernetes at scale, rather than those seeking pure modeling work.

$195k - $345k

We are seeking a Senior Developer Relations Manager to build the developer ecosystem around NVIDIA Dynamo, an open-source framework for distributed AI inference. In this senior individual contributor role, you will help developers deploy and operate inference systems, build practical technical resources, and represent developer needs both in the community and inside NVIDIA. You will build the ecosystem with various inference projects through joint demos, integration guides, upstream contributions, and community programs. You will also develop runnable demos, sample code, technical documentation, and videos to explain Dynamo’s modular ecosystem and demonstrate its data center performance value. Additionally, you will represent the developer community inside NVIDIA by translating feedback into engineering and product requirements, influencing priorities, and communicating progress back to developers. You will advocate externally through technical talks, live demos, workshops, and office hours, helping developers troubleshoot real deployments and contribute to projects.

$184k - $357k

Code Metal is seeking a Staff AI Platform Engineer to architect and build the foundational AI enablement stack that internal engineering teams rely on. This role involves designing and operating production systems for GPU inference serving, model gateways, agent harnesses, context management, and observability. While primarily an engineering role focused on building and operating production systems, it also requires solid data science and AI research fundamentals to work closely with the AI Research team and conduct experiments when necessary. The platform is built to product standards, ensuring reliability and scalability for AI-driven code transpilation, modernization, optimization, modeling, and simulation solutions.

$205k - $250k

Sierra is seeking a Software Engineer for its Inference team to build the systems that power their AI agents. This role focuses on making foundation models fast, reliable, and efficient at scale. You will define the inference architecture, working on systems for serving, routing, capacity management, and optimization for latency, reliability, and cost. This is a systems-first role at the intersection of distributed infrastructure and AI, requiring strong distributed systems fundamentals and experience with complex systems problems.

$230k - $390k

We are a dedicated research lab focused on building, understanding, using, and risk-managing foundation models. Our mission is to advance AI research, cultivate the next generation of AI builders, and make significant contributions to a knowledge-driven economy. As part of our team, you will work on the core of cutting-edge foundation model training alongside world-class researchers, data scientists, and engineers. You will tackle fundamental and impactful challenges in AI development, contributing to groundbreaking AI solutions that have the potential to reshape industries. Strategic and innovative problem-solving skills are essential for establishing our institution as a global hub for high-performance computing in deep learning.

$150k - $450k

Percepta is transforming critical institutions with applied AI, embedding with industry-leading customers to drive AI transformation. We bring together forward-deployed expertise in engineering, product, and research, along with our in-house toolkit for rapidly deploying agentic workflows and strategic partnerships. As a Research Scientist - Reinforcement Learning, you will work at the intersection of RL research and real-world deployment, advancing decision-making capabilities for critical industries. You will collaborate closely with Embedded Product Managers and engineers to ensure our solutions transform how companies operate.

$200k - $400k

Runpod is seeking a Senior ML Systems Engineer focused on Inference to lead efforts in making LLM inference the fastest and most cost-efficient on their platform. This role involves owning LLM serving performance end-to-end, measuring, understanding, and improving it across various models, hardware, and workloads. The work directly impacts customer latency and cost, requiring a hands-on approach to identify and fix bottlenecks, and ensure reliable production deployment. The ideal candidate will be passionate about AI infrastructure and contribute to building the platform for the next generation of AI developers.

$150k - $220k

Ema builds AI Employees for HR, IT, and Finance, automating busy work across the employee experience from recruiting and onboarding to IT support and payroll. Our team, comprised of engineers from top tech companies and graduates from prestigious universities, is backed by leading investors and operates at the forefront of Agentic AI in production. We are developing systems where AI agents learn and improve from their real-world tasks, creating a continuous loop of production data, training, evaluation, and enhanced agent performance. The core challenge lies in determining the most effective interventions to improve agent behavior in live workflows.

As a Senior Engineering Manager, you will lead an engineering team focused on making large language models run faster, cheaper, and more reliably in production. This role balances team leadership with core systems and performance engineering on demanding models in production, requiring you to stay deeply technical and hands-on with the inference stack end to end. The work is applied, not academic, with optimizations landing in real customer deployments. You will tailor deployments to customer needs, take workloads from proof of concept to production, and ensure performance gains deliver clear business and customer value. This leadership role combines people management, technical direction, and hands-on coding and profiling, with a customer-facing component and elements of product and technical solutions leadership.

$250k - $300k

Together AI

Together AI is building the best inference infrastructure for voice applications, powering real-time voice agents and applications with best-in-class latency and reliability for speech-to-text and text-to-speech models. We are seeking a Senior ML Engineer to lead the model serving layer for voice workloads. You will optimize inference engines like TRT-LLM and SGLang to serve models such as Whisper, Parakeet, Orpheus, and Kokoro, focusing on pushing latency and throughput boundaries. This role involves profiling GPU utilization, designing batching strategies for streaming audio, and accelerating the transition of new model architectures from research to production. As a foundational hire on a small, high-impact team, you will address the unique challenges of voice inference, including streaming audio, real-time latency, and tokenization, shaping how Together serves voice models as the industry moves towards end-to-end speech-to-speech systems.

$200k - $260k

Together AI

Together AI's Model Shaping team focuses on tailoring open foundation models for downstream applications, building services that help ML developers select and improve models with domain-specific data. The team also develops efficient model training and evaluation methods, drawing from machine learning, NLP, and ML systems. As a Research Engineer, you will develop a platform for users to customize open-source models with their own data, working across training and inference stacks. You will build and enhance services for Fine-Tuning, Reinforcement Learning, and Evaluation, ensuring a smooth transition from post-training to production serving and optimizing inference engines for RL training. Collaboration with product, research, and engineering teams is key to maintaining API reliability and performance, ultimately contributing to the foundational layer of the open-source AI ecosystem.

$200k - $290k

Crusoe is seeking a Senior Staff Applied AI Inference Engineer to accelerate the abundance of energy and intelligence by optimizing AI infrastructure. This role focuses on making large language models run faster, cheaper, and more reliably in production by owning the inference stack end-to-end. You will be responsible for profiling performance, implementing modern optimization techniques, and diving deep into serving code. The work is applied, directly impacting customer deployments, and involves tailoring solutions to specific models, traffic patterns, latency targets, and cost constraints. You will collaborate with customer engineering teams, guide workloads from proof of concept to production, and ensure engineered gains are realized by users. This is a hands-on engineering role requiring coding, profiling, and low-level optimization, with a customer-facing component and elements of product and technical solutions work.

$250k - $300k

Crusoe is seeking a Staff Applied AI Inference Engineer to accelerate the abundance of energy and intelligence by optimizing large language models for production environments. This role involves owning the inference stack end-to-end, focusing on profiling, applying modern optimization techniques, and deep dives into serving code to enhance speed, reduce cost, and improve reliability. The work is hands-on, involving coding, profiling, and low-level optimization, with a customer-facing component to tailor deployments to specific needs and ensure performance gains are realized in real-world scenarios. You will transition workloads from proof-of-concept to fully monitored production services, making this a core systems and performance engineering role on demanding AI workloads.

$215k - $260k

Nunchux AI is building infrastructure to make multimodal generative AI faster, cheaper, and easier to build on. Founded by MIT PhDs and CMU Professor, the company leverages deep research expertise and production systems experience. The Role focuses on bringing new image and video models to the Modelverse platform, prioritizing speed to launch. You will own the initial integration, coordinate deployment with the cloud team, and work with product engineers on API and Modelverse integration. After launch, you will hand off optimization tasks and maintain the shared code for model integration and serving.

$170k - $240k