Analyzing the Contribution of Top-Down Lexical and Bottom-Up Acoustic Cues in the Detection of Sentence Prominence

Kakouros, Sofoklis; Pelemans, Joris; Verwimp, Lyan; Wambacq, Patrick; Räsänen, Okko

doi:10.21437/interspeech.2016-926

Cited by 7 publications

(5 citation statements)

References 34 publications

(25 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Furthermore, the results from the binary prominence classification provided additional evidence of the importance of the three acoustic correlates of prominence for Dutch. These results are also close to those of an earlier study on the same data [13].…”

Section: Discussionsupporting

confidence: 92%

“…Prominence is a prosodic phenomenon that conveys the subjective impression of emphasis and is defined as the perception of a linguistic unit standing out from its environment (see [4,5,6] for related definitions). Earlier, many studies focused on determining the acoustic correlates of prominence [7,8,9], and, more recently, on methods for its automatic detection [10,11,12,13,14]. One interesting aspect on the study of prominence that has been enabled by the success of deep neural networks (DNNs), and that has not been widely explored, is whether DNNs are capable of learning prominence-like representations of speech.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Prosodic Representations of Prominence Classification Neural Networks and Autoencoders Using Bottleneck Features

Kakouros¹,

Suni²,

Simko³

et al. 2019

Interspeech 2019

Self Cite

View full text Add to dashboard Cite

Prominence perception has been known to correlate with a complex interplay of the acoustic features of energy, fundamental frequency, spectral tilt, and duration. The contribution and importance of each of these features in distinguishing between prominent and non-prominent units in speech is not always easy to determine, and more so, the prosodic representations that humans and automatic classifiers learn have been difficult to interpret. This work focuses on examining the acoustic prosodic representations that binary prominence classification neural networks and autoencoders learn for prominence. We investigate the complex features learned at different layers of the network as well as the 10-dimensional bottleneck features (BNFs), for the standard acoustic prosodic correlates of prominence separately and in combination. We analyze and visualize the BNFs obtained from the prominence classification neural networks as well as their network activations. The experiments are conducted on a corpus of Dutch continuous speech with manually annotated prominence labels. Our results show that the prosodic representations obtained from the BNFs and higher-dimensional non-BNFs provide good separation of the two prominence categories, with, however, different partitioning of the BNF space for the distinct features, and the best overall separation obtained for F0.

show abstract

Section: Discussionsupporting

confidence: 92%

Section: Introductionmentioning

confidence: 99%

Prosodic Representations of Prominence Classification Neural Networks and Autoencoders Using Bottleneck Features

Kakouros¹,

Suni²,

Simko³

et al. 2019

Interspeech 2019

Self Cite

View full text Add to dashboard Cite

show abstract

“…Specifically, it was observed (i) that the tilt measures behave differently for the distinct statistical descriptors, (ii) there are differences in the performance (class separability) for the distinct tilt measures in clean speech, and (iii) noise degradation greatly impacts prominence class separation from around 10 dB SNR, largely diminishing the differences between the estimators. The results also support the finding that tilt is an important correlate for prominence, at least for Dutch (see, e.g., [10,33], see also [34]). Fig.…”

Section: Discussionsupporting

confidence: 87%

Evaluation of Spectral Tilt Measures for Sentence Prominence Under Different Noise Conditions

Kakouros¹,

Räsänen²,

Alku³

2017

Interspeech 2017

Self Cite

View full text Add to dashboard Cite

Spectral tilt has been suggested to be a correlate of prominence in speech, although several studies have not replicated this empirically. This may be partially due to the lack of a standard method for tilt estimation from speech, rendering interpretations and comparisons between studies difficult. In addition, little is known about the performance of tilt estimators for prominence detection in the presence of noise. In this work, we investigate and compare several standard tilt measures on quantifying prominence in spoken Dutch and under different levels of additive noise. We also compare these measures with other acoustic correlates of prominence, namely, energy, F0, and duration. Our results provide further empirical support for the finding that tilt is a systematic correlate of prominence, at least in Dutch, even though energy, F0, and duration appear still to be more robust features for the task. In addition, our results show that there are notable differences between different tilt estimators in their ability to discriminate prominent words from nonprominent ones in different levels of noise.

show abstract

“…The results showed that the best overall feature combination was that of energy, F0, and duration in both corpora whereas combinations of different tilt measures alone and together with energy, F0, and duration, did not seem to improve class separation. Although surprising, earlier studies have also observed little improvements with the addition of tilt measures in supervised classification tasks (see, e.g., Kakouros, Pelemans, Verwimp, Wambacq, & Räsänen, 2016;Streefkerk, Pols, & ten Bosch, 1999). To further examine the effect of tilt in prominence class separation performance, a combination of the best performing basic features (energy, F0, duration) together with the single best performing tilt measures from the source (DNNC6D) and surface (SLF6D) groups were tested and showed no improvements beyond what was achieved with a combination of all basic features.…”

Section: Discussionmentioning

confidence: 91%

Comparison of spectral tilt measures for sentence prominence in speech—Effects of dimensionality and adverse noise conditions

Kakouros

Räsänen

Alku

2018

Speech Communication

Self Cite

View full text Add to dashboard Cite

Linguistic prominence in speech is known to correlate with the acoustic measures of energy, F0, and duration. In contrast, the role of spectral tilt in the realization of prominence has remained more inconsistent between previous empirical investigations. This may be partially due to the lack of a standard method for quantifying spectral tilt or due to difficulties in estimating the acoustical source of spectral tilt, the glottal flow, from continuous speech. These issues have rendered interpretations and comparisons between studies difficult. In addition, (i) little is known about the robustness of tilt estimators for prominence detection in the case when speech is not clean but corrupted, as in real life, by environmental noise or telephone transmission (i.e. degradation caused by bandpass filtering and quantization noise). Moreover, (ii) little attention has been paid to multidimensional representations of source spectrum that can potentially incorporate more information about the phonation style than purely scalar measures. In this work, we study spectral tilt in signaling prominence in spoken Dutch and French under different levels of additive noise, and for telephoneband coded speech, and compare several one-dimensional tilt measures that have been previously encountered in the literature as well as multidimensional tilt measures. We also compare spectral tilt measures with other standard acoustic correlates for prominence, namely, energy, F0, and duration.Our results provide further empirical support for the finding that tilt is a systematic correlate of prominence in Dutch, that the role is smaller in French, and that energy, F0, and duration appear still to be the most robust features for discriminating prominent and non-prominent words. In addition, our results show that there are notable differences between different tilt measures at different levels of noise, and that multidimensional representations for tilt improve class separability from the scalar measures.

show abstract

Analyzing the Contribution of Top-Down Lexical and Bottom-Up Acoustic Cues in the Detection of Sentence Prominence

Cited by 7 publications

References 34 publications

Prosodic Representations of Prominence Classification Neural Networks and Autoencoders Using Bottleneck Features

Prosodic Representations of Prominence Classification Neural Networks and Autoencoders Using Bottleneck Features

Evaluation of Spectral Tilt Measures for Sentence Prominence Under Different Noise Conditions

Comparison of spectral tilt measures for sentence prominence in speech—Effects of dimensionality and adverse noise conditions

Contact Info

Product

Resources

About