Speech Software Engineer
$215k - $235k • Remote • New York • Full-time
Posted 9mo ago
About the job
ASAPP is seeking a Senior Speech Software Engineer to lead the development of both infrastructure and applied speech intelligence for their real-time voice AI platform. This role sits at the intersection of speech research, model optimization, and production engineering, focusing on ensuring ASR and TTS systems meet stringent quality, latency, and reliability standards for enterprise call centers. The engineer will evolve the speech stack to deliver human-like, low-latency voice interactions at massive scale, tuning modern speech models for challenging real-world environments and collaborating closely with Speech Scientists, ML Researchers, and Infrastructure Engineers to integrate cutting-edge technology into robust production systems.
Responsibilities
- Tune and optimize ASR and TTS models for real-world call center environments, improving transcription accuracy, noise robustness, and speaker variability.
- Improve spoken output naturalness by refining prosody, pacing, number and spelling pronunciation, and conversational flow.
- Balance latency vs. quality tradeoffs in streaming speech pipelines to maintain real-time responsiveness.
- Evaluate and integrate emerging speech technologies like noise suppression, voice activity detection, and diarization.
- Architect and modernize a scalable, high-availability voice infrastructure.
- Build multi-threaded, low-latency server frameworks for handling thousands of concurrent real-time audio streams.
- Design and operate streaming ASR → LLM → TTS pipelines for live AI-driven customer conversations.
- Develop robust media stream handling for reliable audio flow.
- Define and implement speech quality evaluation frameworks, including WER/CER analysis and latency tracking.
- Build tooling and dashboards to monitor production performance and detect regressions.
- Create load-testing and simulation tools for high-concurrency voice traffic.
- Partner with Speech Scientists and ML Researchers to productionize new ASR and TTS models.
- Work with Security and Compliance teams to ensure voice data handling meets enterprise and regulatory standards.
- Collaborate with Product teams to translate conversational quality requirements into measurable system improvements.
Requirements
- 5+ years of software engineering experience building and operating production-grade distributed systems.
- Strong proficiency in Golang or Python.
- Experience designing low-latency, high-concurrency systems, ideally involving real-time media or streaming data.
- Practical experience working with ASR and/or TTS systems in applied or production environments.
- Understanding of how to adapt and tune speech models for domain-specific use cases.
- Familiarity with speech quality metrics such as WER, CER, MOS, latency, and streaming stability.
- Strong grasp of audio fundamentals, including sample rates, codecs (Opus, G.711), buffering, packet loss, and jitter.
- Experience evaluating model performance and running structured experiments to improve transcription accuracy and speech naturalness.
- Comfort working with modern ML tooling and model APIs to fine-tune, adapt, or post-process speech model outputs.
- Ability to make pragmatic tradeoffs between model quality, compute cost, and real-time constraints.