Tackling Perception Bias in Unsupervised Phoneme Discovery Using DPGMM-RNN Hybrid Model and Functional Load

Abstract

The human perception of phonemes is biased against speech sounds. The lack of correspondence between perceputal phonemes and acoustic signals forms a big challenge in designing unsupervised algorithms to distinguish phonemes from sound. We propose the DPGMM-RNN hybrid model that improves phoneme categorization by relieving the fragmentation problem. We also merge segments with low functional load, which is the work done by segment contrasts to differentiate between utterances, just like humans who convert unambiguous segments into phonemes as units for immediate perception. Our results show that the DPGMM-RNN hybrid model relieves the fragmentation problem and improves phoneme discriminability. The minimal functional load merge compresses a segment system, preserves information and keeps phoneme discriminability.

Modeling Unsupervised Empirical Adaptation by DPGMM and DPGMM-RNN Hybrid Model to Extract Perceptual Features for Low-Resource ASR

“…Such sensitivity makes DPGMM clustering uncertain for assigning clusters to frames and creates small, random cluster segments inside a phoneme. This is DPGMM's "fragmentation problem" [49].…”

“…In unsupervised phoneme discovery, DPGMM tends to suffers from a fragmentation problem when the model encounters the frames from such acoustically complex phonemes as a fricative with noise-like high frequencies or a vowel with rapid formant transitions [49], [50]. DPGMM tends to generate more clusters than the number of phonemes in any human language [30], [50] when it struggles to discriminate between complex phonemes with higher resolution.…”

“…We propose to use the DPGMM-RNN hybrid model [49], which enhances DPGMM, to model unsupervised empirical adaptation to improve ASR. The DPGMM-RNN hybrid model 1) improves temporal modeling and 2) relieves fragmentation problems of DPGMM with RNN to relearn the connection between acoustic features and DPGMM cluster labels or posterior vectors by listening to feature chunks instead of concentrating on trivial details at the frame level like DPGMM.…”

“…The DPGMM-RNN hybrid model relieved the fragmentation problem and decreased the fragmental level measured by the conditional perplexity [51] and the v-measure [52]. It also reduced the number of clusters of DPGMM [49] and overperformed DPGMM in unsupervised phoneme discovery on datasets from Zerospeech 2019 with an ABX discrimination test at a moderate bitrate [49].…”

“…We analyze the fragmental level of the generated representations from the DPGMM or DPGMM-RNN hybrid model with the conditional perplexity of cluster given phoneme [49] that reflects the average number of cluster types corresponding to one phoneme type. We define the conditional perplexity by the exponential of the conditional entropy [51] of the cluster representation (C) given the phoneme truth (T ) with base 2. where n is the number of frames, n t is the number of frames of phoneme truth t, and n ct is the number of frames annotated as phoneme t and clustered as cluster c. We analyze the discriminability (D) between the representations of a phoneme pair (t 1 and t 2 ) by the KL divergence between the conditional distributions of cluster representation (C) given the phoneme.…”