Multilingual bottle-neck feature learning from untranscribed speech

Chen, Hongjie; Leung, Cheung-Chi; Xie, Lei; Ma, Bin; Li, Haizhou

doi:10.1109/asru.2017.8269009

Cited by 25 publications

(41 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Supervised DNN training requires frame-level labels for all training speech, which could be obtained either via a clustering process or exploiting out-of-domain resources. In [11], [12], DPGMM clustering was performed on conventional short-time spectral features of target speech, followed by multilingual DNN training to obtain the BNF representation. In [13], GMM-universal background model (GMM-UBM) was used to generate frame labels.…”

Section: Related Work a Deep Learning Approaches To Unsupervisementioning

confidence: 99%

“…In [30], BNF representation was generated by applying multi-task learning with both indomain and out-of-domain data [25]. The frame labels for out-of-domain data were obtained by HMM forced alignment, while the labels for in-domain data were from DPGMM clustering [12]. In [5], [14], [31], a DNN AM was trained with transcribed data of an out-of-domain language, and used to extract BNFs or posteriorgrams from target speech.…”

Section: Related Work a Deep Learning Approaches To Unsupervisementioning

confidence: 99%

“…To enable supervised estimation of fMLLRs, clustering results on spectral features were taken as pseudo transcriptions. Chen et al [12] showed that vocal tract length normalization (VTLN) on top of spectral features contribute to generating more robust DPGMM frame labels. In our study, fMLLR features are estimated by exploiting an out-of-domain ASR system.…”

Section: Optimizing Dpgmm Clusteringmentioning

confidence: 99%

“…A DNN model is typically trained using available speech data. The learned features are obtained either from a designated low-dimension hidden layer of the DNN, known as the bottleneck features (BNFs) [12], or from the softmax output layer, known as the posterior features or posteriorgram [13]. To facilitate supervised training of the DNN, target labels of training speech are needed.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Exploiting Cross-Lingual Speaker and Phonetic Diversity for Unsupervised Subword Modeling

Feng

Lee

2019

IEEE/ACM Trans. Audio Speech Lang. Process.

View full text Add to dashboard Cite

This research addresses the problem of acoustic modeling of low-resource languages for which transcribed training data is absent. The goal is to learn robust frame-level feature representations that can be used to identify and distinguish subword-level speech units. The proposed feature representations comprise various types of multilingual bottleneck features (BNFs) that are obtained via multi-task learning of deep neural networks (MTL-DNN). One of the key problems is how to acquire highquality frame labels for untranscribed training data to facilitate supervised DNN training. It is shown that learning of robust BNF representations can be achieved by effectively leveraging transcribed speech data and well-trained automatic speech recognition (ASR) systems from one or more out-of-domain (resourcerich) languages. Out-of-domain ASR systems can be applied to perform speaker adaptation with untranscribed training data of the target language, and to decode the training speech into framelevel labels for DNN training. It is also found that better frame labels can be generated by considering temporal dependency in speech when performing frame clustering. The proposed methods of feature learning are evaluated on the standard task of unsupervised subword modeling in Track 1 of the ZeroSpeech 2017 Challenge. The best performance achieved by our system is 9.7% in terms of across-speaker triphone minimal-pair ABX error rate, which is comparable to the best systems reported recently. Lastly, our investigation reveals that the closeness between target languages and out-of-domain languages and the amount of available training data for individual target languages could have significant impact on the goodness of learned features.

show abstract

Section: Related Work a Deep Learning Approaches To Unsupervisementioning

confidence: 99%

Section: Related Work a Deep Learning Approaches To Unsupervisementioning

confidence: 99%

Section: Optimizing Dpgmm Clusteringmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Exploiting Cross-Lingual Speaker and Phonetic Diversity for Unsupervised Subword Modeling

Feng

Lee

2019

IEEE/ACM Trans. Audio Speech Lang. Process.

View full text Add to dashboard Cite

show abstract

“…Works in [7,16] estimated fMLLR using a pre-trained out-of-domain ASR. Chen et al [8] applied vocal tract length normalization (VTLN). Another direction is to employ DNNs.…”

Section: Introductionmentioning

confidence: 99%

Combining Adversarial Training and Disentangled Speech Representation for Robust Zero-Resource Subword Modeling

2019

View full text Add to dashboard Cite

This study addresses the problem of unsupervised subword unit discovery from untranscribed speech. It forms the basis of the ultimate goal of ZeroSpeech 2019, building text-to-speech systems without text labels. In this work, unit discovery is formulated as a pipeline of phonetically discriminative feature learning and unit inference. One major difficulty in robust unsupervised feature learning is dealing with speaker variation. Here the robustness towards speaker variation is achieved by applying adversarial training and FHVAE based disentangled speech representation learning. A comparison of the two approaches as well as their combination is studied in a DNN-bottleneck feature (DNN-BNF) architecture. Experiments are conducted on ZeroSpeech 2019 and 2017. Experimental results on Ze-roSpeech 2017 show that both approaches are effective while the latter is more prominent, and that their combination brings further marginal improvement in across-speaker condition. Results on ZeroSpeech 2019 show that in the ABX discriminability task, our approaches significantly outperform the official baseline, and are competitive to or even outperform the official topline. The proposed unit sequence smoothing algorithm improves synthesis quality, at a cost of slight decrease in ABX discriminability.

show abstract

Multilingual and unsupervised subword modeling for zero-resource languages

Hermann

Kamper

Goldwater

2021

Computer Speech & Language

View full text Add to dashboard Cite

Unsupervised subword modeling aims to learn lowlevel representations of speech audio in "zero-resource" settings: that is, without using transcriptions or other resources from the target language (such as text corpora or pronunciation dictionaries). A good representation should capture phonetic content and abstract away from other types of variability, such as speaker differences and channel noise. Previous work in this area has primarily focused on learning from target language data only, and has been evaluated only intrinsically. Here we directly compare multiple methods, including some that use only target language speech data and some that use transcribed speech from other (non-target) languages, and we evaluate using two intrinsic measures as well as on a downstream unsupervised word segmentation and clustering task. We find that combining two existing target-language-only methods yields better features than either method alone. Nevertheless, even better results are obtained by extracting target language bottleneck features using a model trained on other languages. Cross-lingual training using just one other language is enough to provide this benefit, but multilingual training helps even more. In addition to these results, which hold across both intrinsic measures and the extrinsic task, we discuss the qualitative differences between the different types of learned features.

show abstract

Multilingual bottle-neck feature learning from untranscribed speech

Cited by 25 publications

References 21 publications

Exploiting Cross-Lingual Speaker and Phonetic Diversity for Unsupervised Subword Modeling

Exploiting Cross-Lingual Speaker and Phonetic Diversity for Unsupervised Subword Modeling

Combining Adversarial Training and Disentangled Speech Representation for Robust Zero-Resource Subword Modeling

Multilingual and unsupervised subword modeling for zero-resource languages

Contact Info

Product

Resources

About