Speech separation of a target speaker based on deep neural networks

Du, Jun; Tu, Yan-Hui; Xu, Yong; Dai, Li-Rong; Lee, Chin‐Hui

doi:10.1109/icosp.2014.7015050

Cited by 65 publications

(48 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The higher the STOI value is, the better the predicted intelligibility is. STOI is a standard metric for evaluating speech separation performance [31], [8], [15]. …”

Section: Results With Speaker-pair Dependent Trainingmentioning

confidence: 99%

“…Their spectral mapping approach substantially improves SNR and objective speech intelligibility. Du et al [8] improved the method in [37] with global variance equailization, dropout training, and noise-aware training strategies. They demonstrated significant improvement over a GMM-based method and good generalization to unseen speakers in testing.…”

Section: Introductionmentioning

confidence: 99%

“…A good choice of input expansion is to select a fixed contextual window that performs the best among several candidate windows. For example, in [14], the masking-based method sets the window length to 3; in [8], the mapping-based method sets the window length to 7. However, different candidate windows may provide complementary information that can further improve the performance.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

A Deep Ensemble Learning Method for Monaural Speech Separation

Zhang

Wang

2016

IEEE/ACM Trans. Audio Speech Lang. Process.

205

102

View full text Add to dashboard Cite

Monaural speech separation is a fundamental problem in robust speech processing. Recently, deep neural network (DNN)-based speech separation methods, which predict either clean speech or an ideal time-frequency mask, have demonstrated remarkable performance improvement. However, a single DNN with a given window length does not leverage contextual information sufficiently, and the differences between the two optimization objectives are not well understood. In this paper, we propose a deep ensemble method, named multicontext networks, to address monaural speech separation. The first multicontext network averages the outputs of multiple DNNs whose inputs employ different window lengths. The second multicontext network is a stack of multiple DNNs. Each DNN in a module of the stack takes the concatenation of original acoustic features and expansion of the soft output of the lower module as its input, and predicts the ratio mask of the target speaker; the DNNs in the same module employ different contexts. We have conducted extensive experiments with three speech corpora. The results demonstrate the effectiveness of the proposed method. We have also compared the two optimization objectives systematically and found that predicting the ideal time-frequency mask is more efficient in utilizing clean training speech, while predicting clean speech is less sensitive to SNR variations.

show abstract

“…The higher the STOI value is, the better the predicted intelligibility is. STOI is a standard metric for evaluating speech separation performance [31], [8], [15]. …”

Section: Results With Speaker-pair Dependent Trainingmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

A Deep Ensemble Learning Method for Monaural Speech Separation

Zhang

Wang

2016

IEEE/ACM Trans. Audio Speech Lang. Process.

205

102

View full text Add to dashboard Cite

show abstract

“…In [17], DNN is adopted as a regression model to predict the log-power spectral features of the target speaker given the input log-power spectral features of mixed speech with acoustic context as shown in Fig. 2.…”

Section: Dnn-1 For Predicting the Targetmentioning

confidence: 99%

“…Other popular approaches include nonnegative matrix factorization (NMF) based model [16]. One recent work [17] uses deep neural networks (DNNs) to solve the separation problem in Eq. (1) in an alternative way.…”

Section: Introductionmentioning

confidence: 99%

Speech separation based on improved deep neural networks with dual outputs of speech features for both target and interfering speakers

et al. 2014

The 9th International Symposium on Chinese Spoken Language Processing

Self Cite

View full text Add to dashboard Cite

In this paper, a novel deep neural network (DNN) architecture is proposed to generate the speech features of both the target speaker and interferer for speech separation. DNN is adopted here to directly model the highly nonlinear relationship between speech features of the mixed signals and the two competing speakers. With the modified output speech features for learning the parameters of the DNN, the generalization capacity to unseen interferers is improved for separating the target speech. Meanwhile, without any prior information from the interferer, the interfering speech can also be separated. Experimental results show that the proposed new DNN enhances the separation performance in terms of different objective measures under the semi-supervised mode where the training data of the target speaker is provided while the unseen interferer in the separation stage is predicted by using multiple interfering speakers mixed with the target speaker in the training stage.

show abstract

Source separation in ecoacoustics: a roadmap towards versatile soundscape information retrieval

Lin

Tsao

2019

Remote Sens Ecol Conserv

View full text Add to dashboard Cite

A comprehensive assessment of ecosystem dynamics requires the monitoring of biological, physical and social changes. Changes that cannot be observed visually may be trackable acoustically through soundscape analysis. Soundscapes vary greatly depending on geophysical events, biodiversity and human activities. However, retrieving source-specific information from geophony, biophony and anthropophony remains a challenging task, due to interference by simultaneous sound sources. Audio source separation is a technique that aims to recover individual sound sources when only mixtures are accessible. Here, we review techniques of monoaural audio source separation with the fundamental theories and assumptions behind them. Depending on the availability of prior information about the source signals, the task can be approached as a blind source separation or a model-based source separation. Most blind source separation techniques depend on assumptions about the behaviour of the source signals, and their performance may deteriorate when the assumptions fail. Model-based techniques generally do not require specific assumptions, and the models are directly learned from labelled data. With the recent advances of deep learning, the model-based techniques can yield state-of-the-art separation performance, accordingly facilitate content-based audio information retrieval. Source separation techniques have been adopted in several ecoacoustic applications to evaluate the contributions from biodiversity and anthropogenic disturbance to soundscape dynamics. They can also be employed as nonlinear filters to improve the recognition of bioacoustic signals. To effectively retrieve ecological information from soundscapes, source separation is a crucial tool. We believe that the future integrations of ecological hypotheses and deep learning can realize a high-performance source separation for ecoacoustics, and accordingly improve soundscape-based ecosystem monitoring. Therefore, we outline a roadmap for applying source separation to assist in soundscape information retrieval and hope to promote cross-disciplinary collaboration. (biophony), their habitats (geophony) and human development (anthropophony) (Dumyahn and Pijanowski 2011; Pijanowski et al. 2011; Sueur and Farina 2015; Krause and Farina 2016). By listening to biotic sounds, the composition of soniferous animals can be characterized and applied to assess wildlife biodiversity (Andr e et al. 2011; Saito et al. 2015; Lin et al. 2017a; Ross et al. 2018). On the other hand, abiotic sounds can be used to evaluate the ecosystem dynamics associated with

show abstract

Speech separation of a target speaker based on deep neural networks

Cited by 65 publications

References 16 publications

A Deep Ensemble Learning Method for Monaural Speech Separation

A Deep Ensemble Learning Method for Monaural Speech Separation

Speech separation based on improved deep neural networks with dual outputs of speech features for both target and interfering speakers

Source separation in ecoacoustics: a roadmap towards versatile soundscape information retrieval

Contact Info

Product

Resources

About