Multi-task joint-learning for robust voice activity detection

Zhuang, Yimeng; Tong, Sibo; Yin, Maofan; Ye, Qian; Yu, Kai

doi:10.1109/iscslp.2016.7918383

Cited by 7 publications

(6 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This paper, thus, confirmed that the MP-aware DNN is useful for replay attack detection. In our future work, we will attempt to apply multitask training [27][28] for DNN. Moreover, we will also make an implementation of i-vector for further performance improvement.…”

Section: Discussionmentioning

confidence: 99%

Exploiting Magnitude and Phase Aware Deep Neural Network for Replay Attack Detection

Phapatanaburi¹,

Buayai

Naktong

et al. 2020

ECTI-EEC

View full text Add to dashboard Cite

Magnitude and phase aware deep neural network (MP aware DNN) based on Fast Fourier Transform information, has recently been received more attention to many speech applications. However, little attention has been paid to its aspect in terms of replay attack detection developed for the automatic speaker verification and countermeasures (ASVspoof 2017). This paper aims to investigate the MP aware DNN as a speech classification for detecting non-replayed (genuine) and replayed speech. Also, to exploit the advantage of the classifier-based complementary to improve the reliable detection decision, we propose a novel method by combining MP aware DNN with standard replay attack detection (that is, the use of constant Q transform cepstral coefficients-based Gaussian mixture model classification: CQCC-based GMM). Experiments are evaluated using ASVspoof 2017 and a standard measure of detection performance called equal error rate (EER). The results showed that MP aware DNN -based detection performed conventional DNN method using only the magnitude/phase features. Moreover, we found that score combination of CQCC-based GMM with MP aware DNN achieved additional improvement, indicating that MP aware DNN is very useful, especially when combined with the CQCC-based GMM for replay attack detection.

show abstract

Section: Discussionmentioning

confidence: 99%

Exploiting Magnitude and Phase Aware Deep Neural Network for Replay Attack Detection

Phapatanaburi¹,

Buayai

Naktong

et al. 2020

ECTI-EEC

View full text Add to dashboard Cite

show abstract

“…Finally, Refs. [ 39 , 40 ] proposes the use of a multi-objective network to jointly train SE and VAD to boost their performance. In this system both modules share the same network with different loss functions.…”

Section: Related Work On Jointly Training Of Speech Enhancement With Different Speech Tasksmentioning

confidence: 99%

Time-Domain Joint Training Strategies of Speech Enhancement and Intent Classification Neural Models

Ali

Falavigna

Brutti

2022

Sensors

View full text Add to dashboard Cite

Robustness against background noise and reverberation is essential for many real-world speech-based applications. One way to achieve this robustness is to employ a speech enhancement front-end that, independently of the back-end, removes the environmental perturbations from the target speech signal. However, although the enhancement front-end typically increases the speech quality from an intelligibility perspective, it tends to introduce distortions which deteriorate the performance of subsequent processing modules. In this paper, we investigate strategies for jointly training neural models for both speech enhancement and the back-end, which optimize a combined loss function. In this way, the enhancement front-end is guided by the back-end to provide more effective enhancement. Differently from typical state-of-the-art approaches employing on spectral features or neural embeddings, we operate in the time domain, processing raw waveforms in both components. As application scenario we consider intent classification in noisy environments. In particular, the front-end speech enhancement module is based on Wave-U-Net while the intent classifier is implemented as a temporal convolutional network. Exhaustive experiments are reported on versions of the Fluent Speech Commands corpus contaminated with noises from the Microsoft Scalable Noisy Speech Dataset, shedding light and providing insight about the most promising training approaches.

show abstract

“…Thus, the input feature is a waveform. In this work, we adopted multitask model of speech enhancement and multi-class voice activity detection (VAD) to further raise the performance of the speech extraction model [25]. VAD head estimates the presence of target and interfering speaker respectively for each time frame from the output of the last dilated convolution block in the time-domain SpeakerBeam structure.…”

Section: Target Speech Extractionmentioning

confidence: 99%

Should We Always Separate?: Switching Between Enhanced and Observed Signals for Overlapping Speech Recognition

et al. 2021

View full text Add to dashboard Cite

Although recent advances in deep learning technology improved automatic speech recognition (ASR), it remains difficult to recognize speech when it overlaps other people's voices. Speech separation or extraction is often used as a front-end to ASR to handle such overlapping speech. However, deep neural network-based speech enhancement can generate 'processing artifacts' as a side effect of the enhancement, which degrades ASR performance. For example, it is well known that single-channel noise reduction for non-speech noise (nonoverlapping speech) often does not improve ASR. Likewise, the processing artifacts may also be detrimental to ASR in some conditions when processing overlapping speech with a separation/extraction method, although it is usually believed that separation/extraction improves ASR. In order to answer the question 'Do we always have to separate/extract speech from mixtures?', we analyze ASR performance on observed and enhanced speech at various noise and interference conditions, and show that speech enhancement degrades ASR under some conditions even for overlapping speech. Based on these findings, we propose a simple switching algorithm between observed and enhanced speech based on the estimated signal-to-interference ratio and signal-to-noise ratio. We demonstrated experimentally that such a simple switching mechanism can improve recognition performance when processing artifacts are detrimental to ASR.

show abstract

Multi-task joint-learning for robust voice activity detection

Cited by 7 publications

References 10 publications

Exploiting Magnitude and Phase Aware Deep Neural Network for Replay Attack Detection

Exploiting Magnitude and Phase Aware Deep Neural Network for Replay Attack Detection

Time-Domain Joint Training Strategies of Speech Enhancement and Intent Classification Neural Models

Should We Always Separate?: Switching Between Enhanced and Observed Signals for Overlapping Speech Recognition

Contact Info

Product

Resources

About