Machine Learning Researcher, Audio

About the job

Responsibilities

• Design and train large-scale text-to-speech (TTS) models for expressive and controllable output.
• Develop neural audio codec-based TTS architectures for efficient, high-fidelity generation.
• Improve prosody modeling, question inflection, emotional expression, and multi-speaker robustness in TTS.
• Optimize TTS systems for real-time, low-latency inference in production.
• Build and fine-tune large-scale automatic speech recognition (ASR) systems robust to various real-world audio conditions.
• Leverage self-supervised pretraining and large-scale weak supervision for ASR.
• Improve transcription accuracy for enterprise scenarios, including structured extraction and conversational nuance.
• Research and implement neural audio codecs for extreme compression with minimal perceptual loss.
• Explore discrete and continuous latent representations for scalable speech modeling.
• Design codec architectures that enable downstream generative modeling and controllable synthesis.
• Curate and process massive audio datasets across languages, speakers, and environments.
• Design staged training curricula and data filtering strategies for large datasets.
• Scale training across distributed GPU clusters, focusing on cost, throughput, and reliability.
• Design ablation studies to isolate the impact of architectural changes.
• Measure improvements using both objective metrics and perceptual evaluations.
• Validate ideas quickly through focused experiments that confirm or eliminate hypotheses.

Requirements

• Experience with self-supervised learning, multimodal modeling, or generative modeling.
• Ability to derive new formulations and implement them efficiently.
• Hands-on experience building or scaling TTS, STT, or neural audio codec systems.
• Familiarity with large-scale speech datasets and real-world audio variability.
• Strong intuition for audio quality, prosody, and conversational dynamics.
• Experience training and serving large models on modern accelerators.
• Knowledge of inference optimization techniques, including quantization, kernel optimization, and memory efficiency.
• Understanding of real-time constraints in telephony or streaming environments.
• Track record of designing controlled experiments and meaningful ablations.
• Comfortable working with both offline benchmarks and live production metrics.
• Ability to move quickly from hypothesis to validation.
• Comfortable in fast-moving startup environments.
• Strong ownership mindset from research through deployment.
• Excited by ambiguous, unsolved problems.
• Experience with large-scale distributed training (bonus).
• Research publications or open-source contributions in speech or language AI (bonus).
• Background in real-time speech systems or telephony (bonus).
• PhD in ML, AI, or a related field, or equivalent research impact (bonus).

Benefits

• Healthcare, dental, vision
• Meaningful equity in a fast-growing company
• Every tool you need to succeed
• Beautiful office in Levi's Plaza, SF with rooftop views