Research Scientist, Speech & Audio
$160k - $185k • Remote • Remote - United States
Posted 23d ago
Job Location
Remote - United States
Tech Stack
Remote Work Policy
Fully remote
Categories
AI Research Engineer
About the job
Innodata is a global data engineering company focused on the intersection of data and Artificial Intelligence (AI). Our mission is to enable the responsible advancement of AI by providing the data, evaluation frameworks, and human expertise needed to build trustworthy AI systems at scale. We offer a range of solutions, platforms, and services for Generative AI builders and adopters, leveraging over 36 years of experience in delivering high-quality data and outstanding customer outcomes.
This role focuses on the science behind speech and audio models, addressing challenges in robustness across accents, noise, and code-switching, speaker diarization, speech naturalness, and streaming latency. You will be instrumental in building the data and evaluation designs that drive progress in these areas for customers and frontier labs. Your work will involve critical judgment on benchmark coverage, transcription conventions, and the appropriate use of automated versus human evaluation metrics. You will collaborate closely with our transcription and linguistics lead to ensure our data standards directly influence model learning.
Responsibilities
- Define how Innodata designs, structures, and evaluates audio data for speech and audio models, and validate these choices experimentally.
- Translate speech and audio model requirements (ASR, text-to-speech, speech-to-speech, diarization, audio-language models, streaming systems) into concrete data specifications, including modalities, transcription/annotation schemas, sampling, and evaluation criteria.
- Develop evaluation methodologies that go beyond word error rate to include semantic accuracy, robustness to noise/accent, code-switching, diarization error rate (DER), naturalness/intelligibility of generated speech, and streaming latency, determining when automated metrics are sufficient.
- Determine how existing and incoming audio should be structured, enriched, and sampled to meet model objectives, considering factors like languages, accents, acoustic conditions, speaker demographics, emotional range, scripted vs. spontaneous speech, and single vs. multi-speaker settings, including low-resource and code-switched speech.
- Collaborate with the transcription and linguistics lead to translate model objectives into transcription specifications and quantify the impact of transcription conventions and quality on ASR and speech model results.
- Partner with the audio solutions and engineering team to ensure collected audio is optimized for model objectives, specifying data requirements.
Requirements
- Experience in speech and audio model development, including ASR, text-to-speech, speech-to-speech, speaker diarization, and audio-language models.
- Understanding of data specifications for speech and audio models, including modalities, transcription schemas, sampling, and evaluation criteria.
- Proficiency in developing evaluation methodologies for speech and audio tasks, including metrics beyond word error rate.
- Knowledge of factors influencing audio data quality and relevance for model training, such as accents, noise, code-switching, speaker demographics, and acoustic conditions.
- Experience collaborating with linguistics and transcription experts to define and implement data standards.
- Familiarity with audio data collection and processing pipelines.