Senior Machine Learning Engineer, Voice AI
$200k - $260k • Remote • San Francisco
Posted 1mo ago
Job Location
San Francisco
Tech Stack
Remote Work Policy
Fully remote
Categories
Machine Learning Engineer
About the job
Together AI is building the best inference infrastructure for voice applications, powering real-time voice agents and applications with best-in-class latency and reliability for speech-to-text and text-to-speech models. We are seeking a Senior ML Engineer to lead the model serving layer for voice workloads. You will optimize inference engines like TRT-LLM and SGLang to serve models such as Whisper, Parakeet, Orpheus, and Kokoro, focusing on pushing latency and throughput boundaries. This role involves profiling GPU utilization, designing batching strategies for streaming audio, and accelerating the transition of new model architectures from research to production. As a foundational hire on a small, high-impact team, you will address the unique challenges of voice inference, including streaming audio, real-time latency, and tokenization, shaping how Together serves voice models as the industry moves towards end-to-end speech-to-speech systems.
Responsibilities
- Own the model serving stack for Together's voice platform across STT, TTS, and speech-to-speech.
- Optimize inference performance for voice models, targeting best-in-class TTFB, throughput, and GPU utilization.
- Productionize voice models on serverless and dedicated endpoints, implementing batching strategies and streaming inference tailored for audio workloads.
- Build and maintain a voice model evaluation framework to measure performance across various conditions.
- Enable new model architectures in the serving stack, including audio-native LLMs and speech-to-speech systems.
- Collaborate with model partners to integrate and optimize their models on Together's infrastructure.
- Profile and debug performance across the full inference stack, shipping measurable improvements.
- Ensure the serving layer meets the latency and reliability requirements of real-time voice APIs.
- Contribute to voice model fine-tuning capabilities for STT and TTS.
- Lay the groundwork for future product development.
Requirements
- 5+ years of experience in ML engineering, focusing on model serving, inference optimization, or ML infrastructure.
- Hands-on experience with LLM serving engines (vLLM, SGLang, TensorRT-LLM, or similar), including modifying engine internals.
- Strong proficiency in Python and PyTorch.
- Experience with GPU profiling and optimization (CUDA, memory management, kernel-level debugging).
- Proven track record of shipping ML systems to production with measurable performance improvements.
- Strong product sense, understanding the needs of developers building voice applications.
- Comfort working in a small, fast-paced, early-stage team environment.
- Experience with speech and audio ML (ASR, TTS architectures, audio signal processing) is a strong plus.
- Familiarity with audio codecs and tokenization schemes (SNAC, Encodec, DAC) is a plus.
- Experience training or fine-tuning speech models is a plus.
- Bachelor's or Master's degree in Computer Science, Electrical Engineering, or related field, or equivalent practical experience.
Benefits
- Competitive compensation
- Startup equity
- Health insurance
- Other competitive benefits