Deep Neural Network Based Multichannel Audio Source Separation

Nugraha, Aditya Arie

doi:10.1007/978-3-319-73031-8_7

Cited by 7 publications

(6 citation statements)

References 61 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As supervised speech source separation techniques, recently, deep neural network (DNN) based approaches with a training dataset This work was done while Yoshiki Masuyama and Yu Nakagome were interns at LINE Corporation. in which there are microphone input signal and corresponding oracle clean data have been widely studied, e.g., deep clustering (DC) [10,11], permutation invariant training (PIT) [12,13], deep attractor network [14,15], and hybrid approaches with BSS [16][17][18]. DNN based approaches can capture complicated spectral characteristics of a speech source.…”

Section: Introductionmentioning

confidence: 99%

“…in which there are microphone input signal and corresponding oracle clean data have been widely studied, e.g., deep clustering (DC) [10,11], permutation invariant training (PIT) [12,13], deep attractor network [14,15], and hybrid approaches with BSS [16][17][18]. DNN based approaches can capture complicated spectral characteristics of a speech source.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Unsupervised Training for Deep Speech Source Separation with Kullback-Leibler Divergence Based Probabilistic Loss Function

Togami

Masuyama

Komatsu

et al. 2020

ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

In this paper, we propose a multi-channel speech source separation with a deep neural network (DNN) which is trained under the condition that no clean signal is available. As an alternative to a clean signal, the proposed method adopts an estimated speech signal by an unsupervised speech source separation with a statistical model. As a statistical model of microphone input signal, we adopts a timevarying spatial covariance matrix (SCM) model which includes reverberation and background noise submodels so as to achieve robustness against reverberation and background noise. The DNN infers intermediate variables which are needed for constructing the timevarying SCM. Speech source separation is performed in a probabilistic manner so as to avoid overfitting to separation error. Since there are multiple intermediate variables, a loss function which evaluates a single intermediate variable is not applicable. Instead, the proposed method adopts a loss function which evaluates the output probabilistic signal directly based on Kullback-Leibler Divergence (KLD). Gradient of the loss function can be back-propagated into the DNN through all the intermediate variables. Experimental results under reverberant conditions show that the proposed method can train the DNN efficiently even when the number of training utterances is small, i.e., 1K.

show abstract

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Unsupervised Training for Deep Speech Source Separation with Kullback-Leibler Divergence Based Probabilistic Loss Function

Togami

Masuyama

Komatsu

et al. 2020

ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

show abstract

“…Nonlinear models, such as deep neural networks (DNNs), are therefore highly applicable because of their ability to identify nonlinear structures in audio signals [11][12][13]. Additionally, recurrent neural networks (RNNs) that exhibit the temporal behaviour of a time sequence can be trained to predict time-frequency masks for target signals and separate sources from a mixed waveform [14].…”

Section: Introductionmentioning

confidence: 99%

Separating Overlapping Bat Calls with a Bi-directional Long Short-term Memory Network

Zhang

Liu

Song

et al. 2020

Preprint

View full text Add to dashboard Cite

Background: Acquiring clear and usable audio recordings is critical for acoustic analysis of animal vocalizations. Bioacoustics studies commonly face the problem of overlapping signals, which can impede the structural identification of vocal units, but the issue is often ignored, as there is currently no satisfactory solution. This study presents a bi-directional long short-term memory (BLSTM) network to separate overlapping bat calls and reconstruct waveforms. The separation quality was evaluated using seven temporal-spectrum parameters. The applicability of this method for bat calls was assessed using six different species. In addition, clustering analysis was conducted with separated echolocation calls from each bat species. Results: All syllables in the overlapping calls were separated with high robustness across species. A comparison between the seven temporal-spectrum parameters showed no significant difference and negligible deviation between the extracted and original calls, indicating high separation quality. Clustering analysis also produced an accuracy of 93.8%, suggesting the reconstructed waveforms could be reliably used. Conclusions: The study extends the application of deep neural network to separate overlapping animal sound. These results suggest the proposed technique is a convenient and automated approach for separating overlapping calls using a BLSTM network. This powerful deep neural network approach has the potential to solve complex problems in bioacoustics.

show abstract

“…First and foremost, Gaussian processes realizations may not explore more than a few standard deviations, which means that Bayesian inference in these models is intrinsically very sensitive to initialization, since the probability mass is almost everywhere negligible. A common workaround is to further constrain covariance models through shallow (Ozerov et al [31]) or deep (Nugraha et al [30]) parametric constraints, but another complementary route is to simply opt for heavy-tail models, for which much more robust inference is possible. For instance, multivariate Laplace filters (Wang et al [41]) were successfully used for robust detection and result from Bayesian inference in a state-space model where some variables are Laplace distributed.…”

Section: Introductionmentioning

confidence: 99%

Separation of alpha-stable random vectors

Fontaine

Badeau

2020

Signal Processing

Self Cite

View full text Add to dashboard Cite

Source separation aims at decomposing a vector into additive components. This is often done by first estimating source parameters before feeding them into a filtering method, often based on ratios of covariances. The whole pipeline is traditionally rooted in some probabilistic framework providing both the likelihood for parameter estimation and the separation method. While Gaussians are ubiquitous for this purpose, many studies showed the benefit of heavy-tailed models for estimation. However, there is no counterpart filtering method to date exploiting such formalism, so that related studies revert to covariance-based filtering after estimation is finished. Here, we introduce a new multivariate separation technique, that fully exploits the flexibility of α-stable heavy-tailed distributions. We show how a spatial representation can be exploited, which decomposes the observation as an infinite sum of contributions originating from all directions. Two methods for separation are derived. The first one is non-linear and similar to a beamforming technique, while the second one is linear, but minimizes a covariation criterion, which is the counterpart of the covariance for α-stable vectors. We evaluate the proposed techniques in a large number of challenging and adverse situations on synthetic experiments, demonstrating their performance for the extraction of signals from strong interferences.

show abstract

Deep Neural Network Based Multichannel Audio Source Separation

Cited by 7 publications

References 61 publications

Unsupervised Training for Deep Speech Source Separation with Kullback-Leibler Divergence Based Probabilistic Loss Function

Unsupervised Training for Deep Speech Source Separation with Kullback-Leibler Divergence Based Probabilistic Loss Function

Separating Overlapping Bat Calls with a Bi-directional Long Short-term Memory Network

Separation of alpha-stable random vectors

Contact Info

Product

Resources

About