research area Accessibility, research area Speech and Natural Language Processingconference Interspeech
content type paperpublished June 2025

Voice Quality Dimensions as Interpretable Primitives for Speaking Style for Atypical Speech and Affect

AuthorsJaya Narain, Vasudha Kowtha†, Colin Lea, Lauren Tooley, Dianna Yee, Vikramjit Mitra, Zifang Huang†, Miquel Espi Marques, Jon Huang, Carlos Avendano, Shirley Ren

View publication

Perceptual voice quality dimensions describe key characteristics of atypical speech and other speech modulations. Here we develop and evaluate voice quality models for seven voice and speech dimensions (intelligibility, imprecise consonants, harsh voice, naturalness, monoloudness, monopitch, and breathiness). Probes were trained on the public Speech Accessibility (SAP) project dataset with 11,184 samples from 434 speakers, using embeddings from frozen pre-trained models as features. We found that our probes had both strong performance and strong generalization across speech elicitation categories in the SAP dataset. We further validated zero-shot performance on additional datasets, encompassing unseen languages and tasks: Italian atypical speech, English atypical speech, and affective speech. The strong zero-shot performance and the interpretability of results across an array of evaluations suggests the utility of using voice quality dimensions in speaking style-related tasks.

  • † Work done while at Apple

Related readings and updates.

Hypernetworks for Personalizing ASR to Atypical Speech

June 14, 2024research area Accessibility, research area Speech and Natural Language Processingconference Transactions of the Association for Computational Linguistics

*Equal Contributors Parameter-efficient fine-tuning (PEFT) for personalizing automatic speech recognition (ASR) has recently shown promise for adapting general population models to atypical speech. However, these approaches assume a priori knowledge of the atypical speech disorder being adapted for -- the diagnosis of which requires expert knowledge that is not always available. Even given this knowledge, data scarcity and high…
Read more

Advancing Speech Accessibility with Personal Voice

December 18, 2023research area Speech and Natural Language Processing

A voice replicator is a powerful tool for people at risk of losing their ability to speak, including those with a recent diagnosis of amyotrophic lateral sclerosis (ALS) or other conditions that can progressively impact speaking ability. First introduced in May 2023 and made available on iOS 17 in September 2023, Personal Voice is a tool that creates a synthesized voice for such users to speak in FaceTime, phone calls, assistive communication apps, and in-person conversations.

Read more