On structured sparsity of phonological posteriors for linguistic parsing

Abstract

The speech signal conveys information on different time scales from short (20-40 ms) time scale or segmental, associated to phonological and phonetic information to long (150-250 ms) time scale or supra segmental, associated to syllabic and prosodic information. Linguistic and neurocognitive studies recognize the phonological classes at segmental level as the essential and invariant representations used in speech temporal organization.In the context of speech processing, a deep neural network (DNN) is an effective computational method to infer the probability of individual phonological classes from a short segment of speech signal. A vector of all phonological class probabilities is referred to as phonological posterior. There are only very few classes comprising a short term speech signal; hence, the phonological posterior is a sparse vector. Although the phonological posteriors are estimated at segmental level, we claim that they convey supra-segmental information. Specifically, we demonstrate that phonological posteriors are indicative of syllabic and prosodic events.Building on findings from converging linguistic evidence on the gestural model of Articulatory Phonology as well as the neural basis of speech perception, we hypothesize that phonological posteriors convey properties of linguistic classes at multiple time scales, and this information is embedded in their support (index) of active coefficients. To verify this hypothesis, we obtain a binary representation of phonological posteriors at the segmental level which is referred to as first-order sparsity structure; the high-order structures are obtained by the concatenation of first-order binary vectors. It is then confirmed that the classification of supra-segmental linguistic events, the problem known as linguistic parsing, can be achieved with high accuracy using a simple binary pattern matching of first-order or high-order structures.

Phonological Posterior Hashing for Query by Example Spoken Term Detection

“…This result was expected due to the binary nature of phonological posteriors [18,21]. Moreover, if the dewarped posteriors are quantized into binary vectors and Jaccard similarity is used for binary pattern matching [21], similar results as the Spearman similarity measure is achieved. This observation again confirms that the space of phonological posteriors is highly structured and the structures bear more information than the exact posterior values.…”

“…The permissible combinations are highly constrained due to articulatory mechanisms governing speech production. Therefore, the probabilities constituting a posterior are confined to a small number of components where the indices of high probabilities determine the unique structure of the vocal machinery in speech production [21]. [1,27].…”

“…The posterior space is highly structured and lowdimensional [18,21]. To exploit this property, we propose to use Spearman's rank correlation to measure the similarity of posterior exemplars.…”

“…We use the open-source DNN based phonological vocoding platform [24] for estimation of the extended Sound Pattern of English (eSPE) phonological posteriors. The motivation for using phonological posteriors is three-fold: (1) Phonological posterior quantization and hashing is found to be effective in search space reduction for accurate classification [14,18,21], (2) Sub-phonetic nature of phonological posteriors facilitates development of flexible and lowresource speech detection and recognition solutions [25], and (3) Phonological posterior are found robust for inter-domain posterior representation where the training and testing acoustic conditions and languages are different [14,18,21].…”

PAoS Markers: Trajectory Analysis of Selective Phonological Posteriors for Assessment of Progressive Apraxia of Speech

“…In addition, we exploit phonological structures [9] to enable automatic analysis of duration and trajectory without any need for automatic alignment. Prior work on phonological structures demonstrate their relation to articulatory postures [9], thus considering the structure of multiple consecutive segments enables quantification of the dynamic and trajectory of articulatory movements and co-articulation. The studies presented in this paper exploit this structural property of phonological posteriors to obtain speech-based markers of PAoS severity.…”