Senior Machine Learning Engineer, Inference
$250k - $300k • Mountain View, California, USA • FullTime
Posted 4h ago
Job Location
Mountain View, California, USA
Tech Stack
Remote Work Policy
On-site
Employment Type
FullTime
Categories
Machine Learning Engineer
About the job
Inworld is a research lab and inference provider focused on real-time AI for consumer-facing applications. We build first-party speech models, serve LLMs, and run the inference behind modular APIs designed for high-volume, real-time workloads. Our models and infrastructure support consumer applications across companions, healthcare, fitness, education, media, and more, serving over 10 trillion LLM tokens per month. We are looking for strong individuals from varied backgrounds who learn fast, thrive in ambiguity, and can demonstrate their experience in building, breaking, and understanding complex systems.
Responsibilities
- Optimize inference for modern serving frameworks and techniques.
- Accelerate models using quantization, distillation, caching, continuous batching, paged attention, and speculative decoding.
- Develop high-performance systems using C++, CUDA, Rust, or optimized Python.
- Profile code and maximize performance on NVIDIA GPUs.
- Implement distributed systems and scaling solutions for multi-GPU/multi-node inference.
- Ensure reliable handling of thousands of concurrent connections.
- Take ownership of models from research to production, including containerization and optimization.
- Contribute to open-source inference engines or create deep-dive technical write-ups.
- Collaborate to solve complex problems and foster a strong team culture.
- Treat performance, latency, and reliability as first-class product features.
Requirements
- Deep understanding of modern serving frameworks and techniques (e.g., vLLM, TRT-LLM).
- Hands-on experience with model acceleration techniques.
- Proficiency in C++, CUDA, Rust, or highly optimized Python.
- Experience with Kubernetes, Ray, custom load balancing, multi-GPU/multi-node inference.
- Experience building backend or ML systems.
- Ability to take a model from research to production.
- Comfortable picking a direction and building the map as you go.
- Bias for impact and shipping stable, performant code.
- Desire to understand the fundamental logic behind decisions.
- Ability to make unclear problems clear.
Benefits
- Bonus
- Equity
- Benefits
- Relocation assistance