Staff / Principal Machine Learning Engineer, Serving - UK
£140k - £200k • UK • FullTime
Posted 5mo ago
Remote Work Policy
On-site
Employment Type
FullTime
Categories
Machine Learning Engineer
About the job
Inworld is seeking a Staff/Principal Machine Learning Engineer to join their research lab focused on building top-ranked real-time voice models. These models power large consumer-facing AI applications across various sectors, reaching hundreds of millions of end-users. The role involves optimizing real-time inference, developing state-of-the-art models, and creating best-in-class APIs and products. The ideal candidate thrives in ambiguity, learns quickly, and can demonstrate a strong track record of building, breaking, and understanding complex systems. This position emphasizes impact, shipping stable and reliable systems, and a deep understanding of the underlying logic behind engineering decisions.
Responsibilities
- Optimize real-time inference for state-of-the-art models.
- Develop and improve serving frameworks and techniques.
- Implement model acceleration strategies such as quantization, distillation, and caching.
- Build and scale high-performance systems using languages like C++, CUDA, or Rust.
- Manage distributed systems and scaling for thousands of concurrent connections.
- Take full-cycle ownership of models from research to production serving.
- Contribute to the development of best-in-class APIs and products.
Requirements
- Deep understanding of modern serving frameworks and techniques (e.g., vLLM, TRT-LLM).
- Hands-on experience with quantization, distillation, caching, continuous batching, paged attention, and speculative decoding.
- Proficiency in C++, CUDA, Rust, or highly optimized Python.
- Ability to profile code and optimize performance on NVIDIA GPUs.
- Experience with Kubernetes, Ray, custom load balancing, multi-GPU/multi-node inference.
- Experience reliably handling thousands of concurrent connections.
- Demonstrated ability to take a model from research to production serving.
- PhD in CS, Physics, Math, or equivalent practical experience in backend or ML systems.
- Legal right to work in the United Kingdom.
Benefits
- Equity
- Benefits
- Full U.S. visa and relocation support may be available for future relocation to the San Francisco Bay Area.