Speech vocoding for laboratory phonology

Abstract

Using phonological speech vocoding, we propose a platform for exploring relations between phonology and speech processing, and in broader terms, for exploring relations between the abstract and physical structures of a speech signal. Our goal is to make a step towards bridging phonology and speech processing and to contribute to the program of Laboratory Phonology. We show three application examples for laboratory phonology: compositional phonological speech modelling, a comparison of phonological systems and an experimental phonological parametric text-to-speech (TTS) system. The featural representations of the following three phonological systems are considered in this work: (i) Government Phonology (GP), (ii) the Sound Pattern of English (SPE), and (iii) the extended SPE (eSPE). Comparing GP- and eSPE-based vocoded speech, we conclude that the latter achieves slightly better results than the former. However, GP - the most compact phonological speech representation - performs comparably to the systems with a higher number of phonological features. The parametric TTS based on phonological speech representation, and trained from an unlabelled audiobook in an unsupervised manner, achieves intelligibility of 85% of the state-of-the-art parametric speech synthesis. We envision that the presented approach paves the way for researchers in both fields to form meaningful hypotheses that are explicitly testable using the concepts developed and exemplified in this paper. On the one hand, laboratory phonologists might test the applied concepts of their theoretical models, and on the other hand, the speech processing community may utilize the concepts developed for the theoretical phonological models for improvements of the current state-of-the-art applications

On the impact of non-modal phonation on phonological features

“…We used the Sound Patterns of English (SPE) feature set [12] for training of the DNNs for phonological posterior estimation. The mapping used to map from phonemes to SPE phonological classes is taken from [20]. The distribution of the phonological labels is nonuniform, driven by mapping different numbers of phonemes to the phonological classes.…”

Perceptual Information Loss due to Impaired Speech Production

“…Shannon proposed this measure 1 In analysis of larger units when co-articulation is important, a binary mapping of co-articulation to the phone attributes may be considered for natural speech production. The numerical evaluation presented in Section V is limited to the (Chomsky's) binary mapping of phoneme and phone attributes [18]. In practice, speech perception enjoys more flexibility and thus, considering a larger set of phoneme and phone attribute mapping presented in natural speech production and perception may be beneficial for speech assessment.…”

“…In the phonological analysis part, phonological posteriors are extracted from the speech signal by DNNs. We used the binary classification of the eSPE set [18], and thus each DNN determines the probability of a particular phonological class.…”

“…We apply DNNs for probabilistic characterization of the phonological classes [17], [18]. We advance the linguistic binary association of the phoneme and phonological classes by considering the dynamic probabilistic associations adapting to the production condition.…”

Sound Pattern Matching for Automatic Prosodic Event Detection

“…English phonological detectors were trained on the WSJ training set si tr s 284 set of 37514 utterance using the extended Sound Pattern of English phonological classes [20]. The phoneme set comprising of 40 phonemes (including "sil", representing silence) was defined by the CMU pronunciation dictionary.…”