Senior / Lead Machine Learning Engineer, Serving - Serbia
Serbia • FullTime
Posted 5mo ago
Remote Work Policy
On-site
Employment Type
FullTime
Categories
Machine Learning Engineer
About the job
Inworld is a research lab building top-ranked real-time voice models used in large consumer-facing AI applications. We are seeking a Senior/Lead Machine Learning Engineer to optimize real-time inference and create best-in-class APIs and products. This role involves tackling complex, ambiguous problems and driving impact through shipped, stable systems. We value engineers who are comfortable with ambiguity, have a bias for action, and obsess over performance, latency, and reliability as core product features.
Responsibilities
- Optimize real-time inference for state-of-the-art models.
- Develop and improve serving frameworks and techniques.
- Implement model acceleration strategies like quantization, distillation, and caching.
- Build and scale high-performance systems for NVIDIA GPUs.
- Manage distributed systems for multi-GPU/multi-node inference and thousands of concurrent connections.
- Take ownership of models from research to production, including containerization and optimization.
- Collaborate with US-based leadership and engineering teams.
Requirements
- Deep understanding of modern serving frameworks (e.g., vLLM, TRT-LLM).
- Hands-on experience with quantization, distillation, caching, continuous batching, paged attention, and speculative decoding.
- Proficiency in C++, CUDA, Rust, or highly optimized Python.
- Experience with profiling code and optimizing for NVIDIA GPUs.
- Experience with distributed systems and scaling (e.g., Kubernetes, Ray).
- Experience with custom load balancing and multi-GPU/multi-node inference.
- Experience reliably handling thousands of concurrent connections.
- Demonstrated ability to take a model from research to production.
- PhD in CS, Physics, Math, or equivalent practical experience in backend or ML systems.
- Professional fluency in English (written and spoken).
Benefits
- Full U.S. visa and relocation support may be available for candidates interested in relocating to the San Francisco Bay Area.