Staff Machine Learning Engineer, Voice AI
$220k - $280k • Remote • San Francisco
Posted 1mo ago
Job Location
San Francisco
Tech Stack
Remote Work Policy
Fully remote
Categories
Machine Learning Engineer
About the job
Together AI is building the best inference infrastructure for voice applications, powering production-grade, real-time voice agents and applications with best-in-class latency and reliability for speech-to-text and text-to-speech models. We are seeking a Staff ML Engineer to lead the model serving layer for voice workloads. This role involves hands-on optimization of inference engines and models, profiling GPU utilization, designing batching strategies for streaming audio, and ensuring rapid deployment of new model architectures from research to production. This is a foundational hire on a small, high-impact team, addressing the unique challenges of voice inference such as streaming audio, tokenization, and real-time latency budgets, and shaping the future of voice model serving.
Responsibilities
- Own the voice inference roadmap end-to-end, defining and executing the technical strategy for optimizing STT, TTS, and speech-to-speech models.
- Drive best-in-class inference performance by architecting and implementing systems targeting leading TTFB, throughput, and GPU utilization for voice workloads.
- Lead productionization of voice models at scale by designing the serving architecture for serverless and dedicated endpoints, including batching strategies, streaming inference pipelines, and memory management tailored to real-time audio.
- Build the voice evaluation platform, designing a rigorous framework covering WER for STT and naturalness/latency for TTS.
- Shape the architecture for next-generation model support, enabling emerging model paradigms like audio-native LLMs and end-to-end speech-to-speech systems.
- Serve as the technical DRI for model partner integrations, leading collaboration with partners and owning the full lifecycle from integration to optimization.
- Diagnose and resolve performance problems in the stack through systematic profiling and root-cause analysis.
- Influence platform architecture across the organization to ensure the serving layer meets the demands of real-time voice APIs.
- Define and scale voice fine-tuning capabilities, leading the technical direction for enabling customer fine-tuning of STT and TTS models.
- Lay technical foundations for a category-defining product surface, architecting systems to support multiple new voice products with minimal rework.
Requirements
- 8+ years of ML engineering experience with a demonstrated focus on model serving, inference optimization, or ML infrastructure at production scale.
- Experience with systems you've owned and scaled in production.