Neural network based regression for robust overlapping speech recognition using microphone arrays

Liu, Weifeng; Dines, John; Magimai.-Doss, Mathew; Bourlard, Hervé

doi:10.21437/interspeech.2008-321

Cited by 4 publications

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

A Neural Network Based Regression Approach for Recognizing Simultaneous Speech

Liu

Kumatani

Dines

et al.

Machine Learning for Multimodal Interaction

Self Cite

View full text Add to dashboard Cite

Abstract. This paper presents our approach for automatic speech recognition (ASR) of overlapping speech. Our system consists of two principal components: a speech separation component and a feature estmation component. In the speech separation phase, we first estimated the speaker's position, and then the speaker location information is used in a GSC-configured beamformer with a minimum mutual information (MMI) criterion, followed by a Zelinski and binary-masking postfilter, to separate the speech of different speakers. In the feature estimation phase, the neural networks are trained to learn the mapping from the features extracted from the pre-separated speech to those extracted from the close-talking microphone speech signal. The outputs of the neural networks are then used to generate acoustic features, which are subsequently used in acoustic model adaptation and system evaluation. The proposed approach is evaluated through ASR experiments on the PASCAL Speech Separation Challenge II (SSC2) corpus. We demonstrate that our system provides large improvements in recognition accuracy compared with a single distant microphone case and the performance of ASR system can be significantly improved both through the use of MMI beamforming and feature mapping approaches.

show abstract

A Neural Network Based Regression Approach for Recognizing Simultaneous Speech

Liu

Kumatani

Dines

et al.

Machine Learning for Multimodal Interaction

Self Cite

View full text Add to dashboard Cite

show abstract

Non-linear mapping for multi-channel speech separation and robust overlapping spech recognition

Liu

Dines

Magimai.-Doss

et al. 2009

2009 IEEE International Conference on Acoustics, Speech and Signal Processing

Self Cite

View full text Add to dashboard Cite

This paper investigates a non-linear mapping approach to extract robust features for ASR and speech separation of overlapping speech. Based on our previous studies, we continue to use two additional sound sources, namely from the target and interfering speakers. The focuses of this work are: 1) We investigate the feature mapping between different domains with the consideration of MMSE criterion and regression optimizations, demonstrating the mapping of log melfilterbank energies to MFCC can be exploited to improve the effectiveness of the regression; 2) We investigate the data-driven filtering for the speech separation by using the mapping method, which can be viewed as a generalized log spectral subtraction and results in better separation performance. We demonstrate the effectiveness of the proposed approach through extensive evaluations on the MONC corpus, which includes both non-overlapping single speaker and overlapping multi-speaker conditions.

show abstract