Models of tone for tonal and non-tonal languages

Metze, Florian; Sheikh, Zaid; Waibel, Alex; Gehring, Jonas; Kilgour, Kevin; Nguyen, Quoc Son; Nguyen, Van Huan

doi:10.1109/asru.2013.6707740

Cited by 30 publications

(22 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We could confirm gains as reported in [24] by including tonal features in our architecture. While training new DBNF networks on the augmented features worked best, integrating the tonal features at the bottleneck level or via glue units improved the resulting acoustic model as well.…”

Section: Discussionsupporting

confidence: 83%

“…Here, we investigate how to integrate fundamental frequency variation (FFV) features [23] into multi-lingual architectures. Recent work demonstrated their suitability for automatic speech recognition, especially when used as input features for neural networks, on a larger version of the Vietnamese corpus used here [24].…”

Section: Extension To Language-specific Input Featuresmentioning

confidence: 99%

“…Even though adding those features was shown to not hurt performance on non-tonal languages [24], this might require the time-consuming retraining of all feature extractors on the modified input feature …”

Section: Extension To Language-specific Input Featuresmentioning

confidence: 99%

See 2 more Smart Citations

DNN acoustic modeling with modular multi-lingual feature extraction networks

Gehring

Nguyen

Metze

et al. 2013

2013 IEEE Workshop on Automatic Speech Recognition and Understanding

Self Cite

View full text Add to dashboard Cite

In this work, we propose several deep neural network architectures that are able to leverage data from multiple languages. Modularity is achieved by training networks for extracting high-level features and for estimating phoneme state posteriors separately, and then combining them for decoding in a hybrid DNN/HMM setup. This approach has been shown to achieve superior performance for single-language systems, and here we demonstrate that feature extractors benefit significantly from being trained as multi-lingual networks with shared hidden representations. We also show that existing mono-lingual networks can be re-used in a modular fashion to achieve a similar level of performance without having to train new networks on multi-lingual data. Furthermore, we investigate in extending these architectures to make use of language-specific acoustic features. Evaluations are performed on a low-resource conversational telephone speech transcription task in Vietnamese, while additional data for acoustic model training is provided in Pashto, Tagalog, Turkish, and Cantonese. Improvements of up to 17.4% and 13.8% over mono-lingual GMMs and DNNs, respectively, are obtained.

show abstract

Section: Discussionsupporting

confidence: 83%

Section: Extension To Language-specific Input Featuresmentioning

confidence: 99%

See 1 more Smart Citation

DNN acoustic modeling with modular multi-lingual feature extraction networks

Gehring

Nguyen

Metze

et al. 2013

2013 IEEE Workshop on Automatic Speech Recognition and Understanding

Self Cite

View full text Add to dashboard Cite

show abstract

“…Therefore, the FBANK feature is used in this study for DNN based acoustic modeling. A recent study showed that with DNN acoustic modeling, the tonal feature (fundamental frequency contour and its related features) helps to improve the performance of ASR system even if the language is not a tonal language ( Metze et al, 2013 ). Therefore, the combination of FBANK and tonal features is also used in this study (hereafter described as the FBANK+tonal feature type).…”

Section: Four Types Of Acoustic Featuresmentioning

confidence: 98%

Combination of multiple acoustic models with unsupervised adaptation for lecture speech transcription

Shen

et al. 2016

Speech Communication

View full text Add to dashboard Cite

“…In our approach, we are learning frame-level speaker-specific embedding features. In other work [95,156], the acoustic features are also augmented by additional information such as prosodic features and signal-to-noise ratios at the frame-level. Nevertheless, our proposed approach is the first approach that uses manifold information as DNN inputs.…”

Section: Connections To Speaker Adaptationmentioning

confidence: 99%

Graph-Based Semisupervised Learning for Acoustic Modeling in Automatic Speech Recognition

Liu

Kirchhoff

2016

IEEE/ACM Trans. Audio Speech Lang. Process.

View full text Add to dashboard Cite

Graph-based semi-supervised learning (SSL) is a widely used semi-supervised learning method in which the labeled data and unlabeled data are jointly represented as a weighted graph, and the information is propagated from the labeled data to the unlabeled data. The key assumption that graph-based SSL makes is that data samples lie on a low dimensional manifold, where samples that are close to each other are expected to have the same class label. More importantly, by exploiting the relationship between training and test samples, graph-based SSL implicitly adapts to the test data.In this thesis, we address several key challenges in applying graph-based SSL to acoustic modeling. We first investigate and compare several state-of-the-art graph-based SSL algorithms on a benchmark dataset. In addition, we propose novel graph construction methods that allow graph-based SSL to handle variable-length input features. We next investigate the efficacy of graph-based SSL in context of a fully-fledged DNN-based ASR system. We compare two different integration frameworks for graph-based learning. First, we propose a lattice-based late integration framework that combines graph-based SSL with the DNN-based acoustic modeling and evaluate the framework on continuous word recognition tasks. Second, we propose an early integration framework using neural graph embeddings and compare two different neural graph embedding features that capture the information of the manifold at different levels. The embedding features are used as input to a DNN system and are shown to outperform the conventional acoustic feature inputs on several medium-to-large vocabulary conversational speech recognition tasks.

show abstract

Models of tone for tonal and non-tonal languages

Cited by 30 publications

References 15 publications

DNN acoustic modeling with modular multi-lingual feature extraction networks

DNN acoustic modeling with modular multi-lingual feature extraction networks

Combination of multiple acoustic models with unsupervised adaptation for lecture speech transcription

Graph-Based Semisupervised Learning for Acoustic Modeling in Automatic Speech Recognition

Contact Info

Product

Resources

About