Senior / Lead Machine Learning Engineer, Serving - Germany
Germany • FullTime
Posted 5mo ago
Remote Work Policy
On-site
Employment Type
FullTime
Categories
Machine Learning Engineer
About the job
Inworld is seeking a Senior / Lead Machine Learning Engineer to join our team in Germany. We are a research lab building top-ranked real-time voice models used in large-scale AI applications across various sectors. Our work involves research, development, optimizing real-time inference, and creating best-in-class APIs and products. We are looking for fast learners with a strong ability to build, break, and understand systems, who thrive in ambiguity and can demonstrate their past work. This role emphasizes full-cycle ownership, taking models from research to production-ready serving.
Responsibilities
- Optimize model inference using modern serving frameworks and techniques.
- Accelerate models through quantization, distillation, caching, continuous batching, paged attention, and speculative decoding.
- Develop high-performance systems in C++, CUDA, Rust, or optimized Python.
- Profile code and optimize performance on NVIDIA GPUs.
- Implement distributed systems and scaling solutions using Kubernetes, Ray, and custom load balancing.
- Manage multi-GPU/multi-node inference and handle thousands of concurrent connections.
- Take ownership of models from research to production, including containerization and serving optimization.
- Collaborate daily with US-based leadership and engineering teams.
Requirements
- Deep understanding of modern serving frameworks and techniques (e.g., vLLM, TRT-LLM).
- Hands-on experience with model acceleration techniques.
- Proficiency in C++, CUDA, Rust, or highly optimized Python.
- Experience with profiling code and optimizing GPU performance.
- Experience with distributed systems and scaling (Kubernetes, Ray, load balancing, multi-GPU/multi-node inference).
- Experience handling thousands of concurrent connections.
- Demonstrated experience with non-trivial systems programming projects, open-source contributions, or technical write-ups.
- Ability to take a model from research to production-ready serving.
- PhD in CS, Physics, Math, or equivalent practical experience building backend or ML systems.
- Professional fluency in English (written and spoken).
Benefits
- Full U.S. visa and relocation support may be available for candidates interested in relocating to the San Francisco Bay Area in the future, subject to business needs and legal requirements.