Multi-task joint-learning of deep neural networks for robust speech recognition

Ye, Qian; Yin, Maofan; You, Yongbin; Yu, Kai

doi:10.1109/asru.2015.7404810

Cited by 22 publications

(17 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The work in Tang et al [Tang, Li and Wang (2016)] integrated speaker recognition and speech recognition into a multi-task learning framework using a recursive structure which attempts to use a unified model to simultaneously identify the two work. The work in Qian et al [Qian, Yin, You et al (2015)] combined two different DNNs (one for feature denoising and one for acoustic modeling) into a complete multi-task framework, in which all parameters can be used in real multi-task mode with two criteria training from scratch. The work in [Thanda and Venkatesan (2017)] combined the speaker's lip visual information with the audio input for speech recognition to learn the mapping of an audio-visual fusion feature and the frame label obtained from the GMM/HMM acoustic model, in which the secondary task is mapping visual features to frame labels derived from another GMM/HMM model.…”

Section: Related Workmentioning

confidence: 99%

Tibetan Multi-dialect Speech and Dialect Identity Recognition

Zhao¹,

Yue²,

Song³

et al. 2019

Computers, Materials &Amp; Continua

View full text Add to dashboard Cite

Tibetan language has very limited resource for conventional automatic speech recognition so far. It lacks of enough data, sub-word unit, lexicons and word inventories for some dialects. And speech content recognition and dialect classification have been treated as two independent tasks and modeled respectively in most prior works. But the two tasks are highly correlated. In this paper, we present a multi-task WaveNet model to perform simultaneous Tibetan multi-dialect speech recognition and dialect identification. It avoids processing the pronunciation dictionary and word segmentation for new dialects, while, in the meantime, allows training speech recognition and dialect identification in a single model. The experimental results show our method can simultaneously recognize speech content for different Tibetan dialects and identify the dialect with high accuracy using a unified model. The dialect information used in output for training can improve multi-dialect speech recognition accuracy, and the low-resource dialects got higher speech content recognition rate and dialect classification accuracy by multi-dialect and multi-task recognition model than task-specific models.

show abstract

Section: Related Workmentioning

confidence: 99%

Tibetan Multi-dialect Speech and Dialect Identity Recognition

Zhao¹,

Yue²,

Song³

et al. 2019

Computers, Materials &Amp; Continua

View full text Add to dashboard Cite

show abstract

“…This ensures that the noise types encountered in the test utterances were unseen during the training phase. We created two multi-conditioned training set by adding 100 types of environmental noises [13], and 11 types of noises from Noisex noise database [14] with the SNR of 0-15 dB with 5 dB of increment, following [15]. We removed the babel noise from Noisex since it is present in the test set.…”

Section: Database Descriptionmentioning

confidence: 99%

“…While the model with direct enhancement block without T-F masking gives 11.20% WER. It has been shown in [15] that by simply adding more layers to DNN baseline does not improve the ASR performance significantly. The results from the model with direct enhancement block seems to confirm this.…”

Section: Network Architecture and Trainingmentioning

confidence: 99%

Label Driven Time-Frequency Masking for Robust Continuous Speech Recognition

Soni

Panda

2019

Interspeech 2019

View full text Add to dashboard Cite

The application of Time-Frequency (T-F) masking based approaches for Automatic Speech Recognition has been shown to provide significant gains in system performance in the presence of additive noise. Such approaches give performance improvement when the T-F masking front-end is trained jointly with the acoustic model. However, such systems still rely on a pre-trained T-F masking enhancement block, trained using pairs of clean and noisy speech signals. Pre-training is necessary due to large number of parameters associated with the enhancement network. In this paper, we propose a flat-start joint training of a network that has both a T-F masking based enhancement block and a phoneme classification block. In particular, we use fully convolutional network as an enhancement front-end to reduce the number of parameters. We train the network by jointly updating the parameters of both these blocks using tied Context-Dependent phoneme states as targets. We observe that pretraining of the proposed enhancement block is not necessary for the convergence. In fact, the proposed flat-start joint training converges faster than the baseline multi-condition trained model. The experiments performed on Aurora-4 database show 7.06% relative improvement over multi-conditioned baseline. We get similar improvements for unseen test conditions as well.

show abstract

“…Therefore, a lot studies has been achieved on introducing techniques evolved for them to every other. These include speaker edition, speaker adaptive education and commonplace historical past version for SRE [1] [2].…”

Section: Introductionmentioning

confidence: 99%

Nexus DNN for Speech and Speaker Recognition

Prakash,

Devi

2019

IJEAT

View full text Add to dashboard Cite

Over the years, many efforts have been made on improving recognition accuracies on Automatic speech recognition (ASR) and speaker recognition (SRE), and many different technologies have been developed. Given the close relationship between these two tasks, researchers have proposed different ways to introduce techniques developed for these tasks to each other . In this paper an open source experimental framework is proposed for speech and speaker recognition. Then a unified model, Nexus-DNN is developed that is trained jointly for speech and speaker recognition. Experimental results show that the combined model can effectively perform ASR and SRE tasks.

show abstract

Multi-task joint-learning of deep neural networks for robust speech recognition

Cited by 22 publications

References 18 publications

Tibetan Multi-dialect Speech and Dialect Identity Recognition

Tibetan Multi-dialect Speech and Dialect Identity Recognition

Label Driven Time-Frequency Masking for Robust Continuous Speech Recognition

Nexus DNN for Speech and Speaker Recognition

Contact Info

Product

Resources

About