Multitask Feature Learning for Low-Resource Query-by-Example Spoken Term Detection

Chen, Hongjie; Leung, Cheung-Chi; Xie, Lei; Ma, Bin; Li, Haizhou

doi:10.1109/jstsp.2017.2764270

Cited by 13 publications

(4 citation statements)

References 38 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In [5], [14], language-mismatched ASR systems were utilized to decode the target speech, and frame labels were generated from the ASR decoding lattices. In [30], BNF representation was generated by applying multi-task learning with both indomain and out-of-domain data [25]. The frame labels for out-of-domain data were obtained by HMM forced alignment, while the labels for in-domain data were from DPGMM clustering [12].…”

Section: Related Work a Deep Learning Approaches To Unsupervisementioning

confidence: 99%

Exploiting Cross-Lingual Speaker and Phonetic Diversity for Unsupervised Subword Modeling

Feng

Lee

2019

IEEE/ACM Trans. Audio Speech Lang. Process.

View full text Add to dashboard Cite

This research addresses the problem of acoustic modeling of low-resource languages for which transcribed training data is absent. The goal is to learn robust frame-level feature representations that can be used to identify and distinguish subword-level speech units. The proposed feature representations comprise various types of multilingual bottleneck features (BNFs) that are obtained via multi-task learning of deep neural networks (MTL-DNN). One of the key problems is how to acquire highquality frame labels for untranscribed training data to facilitate supervised DNN training. It is shown that learning of robust BNF representations can be achieved by effectively leveraging transcribed speech data and well-trained automatic speech recognition (ASR) systems from one or more out-of-domain (resourcerich) languages. Out-of-domain ASR systems can be applied to perform speaker adaptation with untranscribed training data of the target language, and to decode the training speech into framelevel labels for DNN training. It is also found that better frame labels can be generated by considering temporal dependency in speech when performing frame clustering. The proposed methods of feature learning are evaluated on the standard task of unsupervised subword modeling in Track 1 of the ZeroSpeech 2017 Challenge. The best performance achieved by our system is 9.7% in terms of across-speaker triphone minimal-pair ABX error rate, which is comparable to the best systems reported recently. Lastly, our investigation reveals that the closeness between target languages and out-of-domain languages and the amount of available training data for individual target languages could have significant impact on the goodness of learned features.

show abstract

Section: Related Work a Deep Learning Approaches To Unsupervisementioning

confidence: 99%

Exploiting Cross-Lingual Speaker and Phonetic Diversity for Unsupervised Subword Modeling

Feng

Lee

2019

IEEE/ACM Trans. Audio Speech Lang. Process.

View full text Add to dashboard Cite

show abstract

“…Enhanced phone posteriors and phonological posteriors have also been used as speech representation [14], [15]. Both supervised and unsupervised bottleneck features from DNNs have been used for query detection [5], [6], [16].…”

Section: Prior Workmentioning

confidence: 99%

Neural Network based End-to-End Query by Example Spoken Term Detection

Ram¹,

Miculicich²,

Bourlard³

2019

Preprint

View full text Add to dashboard Cite

This paper focuses on the problem of query by example spoken term detection (QbE-STD) in zero-resource scenario. State-of-the-art approaches primarily rely on dynamic time warping (DTW) based template matching techniques using phone posterior or bottleneck features extracted from a deep neural network (DNN). We use both monolingual and multilingual bottleneck features, and show that multilingual features perform increasingly better with more training languages. Previously, it has been shown that the DTW based matching can be replaced with a CNN based matching while using posterior features. Here, we show that the CNN based matching outperforms DTW based matching using bottleneck features as well. In this case, the feature extraction and pattern matching stages of our QbE-STD system are optimized independently of each other. We propose to integrate these two stages in a fully neural network based end-to-end learning framework to enable joint optimization of those two stages simultaneously. The proposed approaches are evaluated on two challenging multilingual datasets: Spoken Web Search 2013 and Query by Example Search on Speech Task 2014, demonstrating in each case significant improvements.

show abstract

“…According to the John Dewey's education theory [1], the "learning by doing" approach takes the students as the learning task center. Compared with traditional teaching, it is helpful to mobilize students' learning enthusiasm and get them to participate in the process of teaching and learning, therefore it promotes the integration of the curriculum and fits today's STEM (Science, Technology, Engineering and Mathematics) education thought [13][14][15][16][17][18]. In the task-driven classroom teaching method, we construct suitable learning tasks, measures, requirements, learning environment and conditions, and these guide students to find the best way to solve the problem [7].…”

Section: Research Purposementioning

confidence: 99%

Effects of Using Task-Driven Classroom Teaching on Students’ Learning Attitudes and Learning Effectiveness in an Information Technology Course

Liu

2018

Sustainability

View full text Add to dashboard Cite

The Internet era of the modern day means that information technology no longer depends on computer theory. Instead, new education methods are used for innovative application. Therefore, the task-driven classroom teaching method is integrated into information technology teaching of junior middle school by designing specific cases. From the theoretical knowledge teaching and the comprehensive operational practice teaching, we explore the effect of using the task-driven classroom learning method on students’ learning attitudes and learning effectiveness in an information technology course. The experimental results show that using the task-driven classroom teaching method has significant positive correlation on student learning performance, learning interest, and creative thinking. This idea can stimulate interest in learning and improve the comprehensive quality of students; thus, it can promote the reform of information technology education and curriculums. For a more detailed discussion, the results will be discussed in this paper.

show abstract

Multitask Feature Learning for Low-Resource Query-by-Example Spoken Term Detection

Cited by 13 publications

References 38 publications

Exploiting Cross-Lingual Speaker and Phonetic Diversity for Unsupervised Subword Modeling

Exploiting Cross-Lingual Speaker and Phonetic Diversity for Unsupervised Subword Modeling

Neural Network based End-to-End Query by Example Spoken Term Detection

Effects of Using Task-Driven Classroom Teaching on Students’ Learning Attitudes and Learning Effectiveness in an Information Technology Course

Contact Info

Product

Resources

About