Speech Structure and Its Application to Robust Speech Processing

Abstract

Speech communication consists of three steps: production, transmission, and hearing. Every step inevitably involves acoustic distortions due to gender differences, age, microphone-and room-related factors, and so on. In spite of these variations, listeners can extract linguistic information from speech as easily as if the communications had not been affected by variations at all. One may hypothesize that listeners modify their internal acoustic models whenever extralinguistic factors change. Another possibility is that the linguistic information in speech can be represented separately from the extralinguistic factors. In this study, being inspired by studies of humans and animals, a novel solution to the problem of intrinsic variations is proposed. Speech structures invariant to these variations are derived as transform-invariant features and their linguistic validity is discussed. Their high robustness is demonstrated by applying the speech structures to automatic speech recognition and pronunciation proficiency estimation. This paper also describes the immaturity of the current implementation and application of speech structures.

Automatic pronunciation clustering using a World English archive and pronunciation structure analysis

“…Minematsu et al proposed a new method of representing speech, called speech structure, and proved that the acoustic variations, corresponding to any linear transformation in the cepstrum domain, can be effectively unseen in the representation [9]. This invariance is due to the invariance of the Bhattacharyya distance (BD), which is calculated using equation 2 and is proved to be invariant with any linear transform.…”

“…The BD is calculated from any pair of distributions and the resulting full set of the BDs forms an invariant distance matrix. This ma- Fig.6 Speaker-independent pronunciation structure Fig.7 Inter-speaker structure difference [12] trix-based representation of an utterance is called pronunciation structure [9]. The structure only represents the local and global contrastive aspects of a given utterance, which is theoretically similar to Jakobson's structural phonology [10].…”

“…We also consider that our system can become more comparable to the perfect recognizer by tuning input features and regression methods. For features, we can use Multiple Stream Structuralization (MSS) [9] and, as discussed in [12], use of absolute features in addition to contrast (relational) features will also be effective to improve the performance. For regression, we're interested in applying kNN-SVR [25] to our task.…”

“…To this end, we use pronunciation structure analysis for feature extraction and we also use support vector regression for distance prediction. The invariant structure analysis was proposed in [8,9] inspired by Jakobson's structural phonology [10] and it can extract invariant and robust features. The structural features were already introduced to various tasks such as pronunciation scoring [11,12], pronunciation error detection [13], language learners clustering [14], dialect analysis [15], and automatic speech recognition [16,17,18].…”

Automatic Chinese pronunciation error detection using SVM trained with structural features

“…Recently, a novel structural model of pronunciation was proposed [6], which works effectively to remove the nonlinguistic aspects of speech from speech acoustics and keep the linguistic aspects well at the same time. Since the nonlinguistic change of speech features is often modeled as feature transformation, the novel model is based on completely transform-invariant features, which is f-divergence [7].…”