An Acoustic Measure for Word Prominence in Spontaneous Speech

Abstract

An algorithm for automatic speech prominence detection is reported in this paper. We describe a comparative analysis on various acoustic features for word prominence detection and report results using a spoken dialog corpus with manually assigned prominence labels. The focus is on features such as spectral intensity and speech rate that are directly extracted from speech based on a correlation-based approach without requiring explicit linguistic or phonetic knowledge. Additionally, various pitch-based measures are studied with respect to their discriminating ability for prominence detection. A parametric scheme for modeling pitch plateau is proposed and this feature alone is found to outperform the traditional local pitch statistics. Two sets of experiments are used to explore the usefulness of the acoustic score generated using these features. The first set focuses on a more traditional way of word prominence detection based on a manually-tagged corpus. A 76.8% classification accuracy was achieved on a corpus of role-playing spoken dialogs. Due to difficulties in manually tagging speech prominence into discrete levels (categories), the second set of experiments focuses on evaluating the score indirectly. Specifically, through experiments on the Switchboard corpus, it is shown that the proposed acoustic score can discriminate between content word and function words in a statistically significant way. The relation between speech prominence and content/function words is also explored. Since prominent words tend to be predominantly content words, and since content words can be automatically marked from text-derived part of speech (POS) information, it is shown that the proposed acoustic score can be indirectly cross-validated through POS information.

Audio-Visual Prosody: Perception, Detection, and Synthesis of Prominence

“…decoding in speech recognition, and hence can be used for syntactic parsing [22], speech comprehension [23], and more recently, research is focusing on the audio-visual function of prosody, for example, in [24] it was found that visual prominence can enhance speech comprehension if coupled with the acoustic. Hence, it is important for speech driven avatars to detect prominent segments in the speech signal to drive gestures.…”

Focused word spotting in spoken Korean based on fundamental frequency

“…Nonverbal information, such as the volume and speed of speech, gestures, pauses, facial expressions, etc., is an important factor to fully understand dialogs or communications [1,2,3]. Nonetheless, after speech signal is converted into text by a speech recognition system, only the language information remains and is utilized for comprehending the meaning of the speech.…”

“…Kim and Scassellati made use of prosodic features of English for robots to learn behavior [7]. Wang and Narayanan studied sentence boundary detection and key information extraction by measuring the speed of English utterances [3,8]. S.-A.…”

Use of syllable nuclei locations to improve ASR

“…This idea was expanded on by Wang to include temporal correlation and a number of other improvements [3,4], and this is the method employed in this work. Wang used this detection method in [12] to create speaking rate and syllable length features for automatic speech prominence detection.…”