3PRO – An unsupervised method for the automatic detection of sentence prominence in speech

Kakouros, Sofoklis; Räsänen, Okko

doi:10.1016/j.specom.2016.06.004

Cited by 26 publications

(22 citation statements)

References 50 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For the unsupervised system and for each fold, five orders of the acoustic n-gram models (n = 1, 2, 3, 4, and 5) were trained for energy, F0, and spectral tilt, on speech data from 28 speakers, always keeping 1 (out of 10) of the annotated speakers for evaluation. In order to evaluate performance for different threshold levels, hyperparameter λ was varied between [-2, 2] with steps of 0.05 for the lexical, acoustic, and combined models (see [10,35] for examples on the effect of λ). Table 1 presents the results for the independent features and the most relevant combinations, as well as the combined model (acoustic+lexical) performance.…”

Section: Resultsmentioning

confidence: 99%

See 1 more Smart Citation

Analyzing the Contribution of Top-Down Lexical and Bottom-Up Acoustic Cues in the Detection of Sentence Prominence

Kakouros¹,

Pelemans²,

Verwimp³

et al. 2016

Interspeech 2016

Self Cite

View full text Add to dashboard Cite

Recent work has suggested that prominence perception could be driven by the predictability of the acoustic prosodic features of speech. On the other hand, lexical predictability and part of speech information are also known to correlate with prominence. In this paper, we investigate how the bottom-up acoustic and top-down lexical cues contribute to sentence prominence by using both types of features in unsupervised and supervised systems for automatic prominence detection. The study is conducted using a corpus of Dutch continuous speech with manually annotated prominence labels. Our results show that unpredictability of speech patterns is a consistent and important cue for prominence at both the lexical and acoustic levels, and also that lexical predictability and part-of-speech information can be used as efficient features in supervised prominence classifiers.

show abstract

Section: Resultsmentioning

confidence: 99%

“…For KNN, the number of nearest neighbors was set to k = 13 since this provided the most consistent performance in preliminary testing. SVMs used radial basis function with a scale factor of σ = 12.08 and box constraint C = 100, as these were previously optimized in the context of acoustic features with the same data [35].…”

Section: Supervised Classificationmentioning

confidence: 99%

Analyzing the Contribution of Top-Down Lexical and Bottom-Up Acoustic Cues in the Detection of Sentence Prominence

Kakouros¹,

Pelemans²,

Verwimp³

et al. 2016

Interspeech 2016

Self Cite

View full text Add to dashboard Cite

show abstract

“…In support of the acoustic predictability hypothesis, we have earlier shown that unpredictability of acoustic prosodic trajectories is highly correlated with human judgments of prominence on the same input in several different languages (Kakouros & Räsänen, 2014, 2016a, 2016b and that both prosodic and lexical predictability contribute to the prominence of words (Kakouros, Pelemans, Vervimp, Wambacq & Räsänen, 2016), effectively replacing the signal-based component of Cole et al (2010) with a fully probabilistic model at multiple levels of representation. The hypothesis also receives support from electrophysiological studies.…”

Section: Introductionmentioning

confidence: 84%

Making predictable unpredictable with style – Behavioral and electrophysiological evidence for the critical role of prosodic expectations in the perception of prominence in speech

2018

Self Cite

View full text Add to dashboard Cite

Perceptual prominence of linguistic units such as words has been earlier connected to the concepts of predictability and attentional orientation. One hypothesis is that low-probability prosodic or lexical content is perceived as prominent due to the surprisal and high information value associated with the stimulus. However, the existing behavioral studies have used stimulus manipulations that follow or violate typical linguistic patterns present in the listeners' native language, i.e., assuming that the listeners have already established a model for acceptable prosodic patterns in the language. In the present study, we investigated whether prosodic expectations and the resulting subjective impression of prominence is affected by brief statistical adaptation to suprasegmental acoustic features in speech, also in the case where the prosodic patterns do not necessarily follow languagetypical marking for prominence. We first exposed listeners to five minutes of speech with uneven distributions of falling and rising fundamental frequency (F0) trajectories on sentence-final words, and then tested their judgments of prominence on a set of new utterances. The results show that the probability of the F0 trajectory affects the perception of prominence, a less frequent F0 trajectory making a word more prominent independently of the absolute direction of F0 change. In the second part of the study, we conducted EEG-measurements on a set of new subjects listening to similar utterances with predominantly rising or falling F0 on sentence-final words. Analysis of the resulting event-related potentials (ERP) reveals a significant difference in N200 and N400 ERP-component amplitudes between standard and deviant prosody, again independently of the F0 direction and the underlying lexical content. Since N400 has earlier been associated with semantic processing of stimuli, this suggests that listeners implicitly track probabilities at the suprasegmental level and that predictability of a prosodic pattern during a word has an impact to the semantic processing of the word. Overall, the study suggests that prosodic markers for prominence are at least partially driven by the statistical structure of recently perceived speech, and therefore prominence perception could be based on statistical learning mechanisms similar to those observed in early word learning, but in this case operating at the level of suprasegmental acoustic features.

show abstract

“…Specifically, the manually labeled prominence markings were used to divide the data into two categories: prominent and non-prominent words. As the data have been labeled by two annotators, all words with at least one prominence marking were considered as prominent (see [32] for a similar approach). For the evaluation, five wordlevel statistical descriptors were computed for all measured features: (i) mean, (ii) max, (iii) min, (iv) standard deviation (SD), and (v) the feature range during the word.…”

Section: Discussionmentioning

confidence: 99%

Evaluation of Spectral Tilt Measures for Sentence Prominence Under Different Noise Conditions

Kakouros¹,

Räsänen²,

Alku³

2017

Interspeech 2017

Self Cite

View full text Add to dashboard Cite

Spectral tilt has been suggested to be a correlate of prominence in speech, although several studies have not replicated this empirically. This may be partially due to the lack of a standard method for tilt estimation from speech, rendering interpretations and comparisons between studies difficult. In addition, little is known about the performance of tilt estimators for prominence detection in the presence of noise. In this work, we investigate and compare several standard tilt measures on quantifying prominence in spoken Dutch and under different levels of additive noise. We also compare these measures with other acoustic correlates of prominence, namely, energy, F0, and duration. Our results provide further empirical support for the finding that tilt is a systematic correlate of prominence, at least in Dutch, even though energy, F0, and duration appear still to be more robust features for the task. In addition, our results show that there are notable differences between different tilt estimators in their ability to discriminate prominent words from nonprominent ones in different levels of noise.

show abstract

3PRO – An unsupervised method for the automatic detection of sentence prominence in speech

Cited by 26 publications

References 50 publications

Analyzing the Contribution of Top-Down Lexical and Bottom-Up Acoustic Cues in the Detection of Sentence Prominence

Analyzing the Contribution of Top-Down Lexical and Bottom-Up Acoustic Cues in the Detection of Sentence Prominence

Making predictable unpredictable with style – Behavioral and electrophysiological evidence for the critical role of prosodic expectations in the perception of prominence in speech

Evaluation of Spectral Tilt Measures for Sentence Prominence Under Different Noise Conditions

Contact Info

Product

Resources

About