Informed Source Separation Using Iterative Reconstruction

Sturmel, Nicolas; Daudet, Laurent

doi:10.1109/tasl.2012.2215597

Cited by 24 publications

(13 citation statements)

References 15 publications

(30 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To improve the consistency, one stream of research is focused on iterative methods such as the classic Griffin-Lim algorithm [14], multiple input spectrogram inverse (MISI) [15], ISSIR [16], and consistent Wiener filtering [17], which can recover the clean phase to some extent starting from the mixture phase and a good estimated magnitude by iteratively performing STFT and iSTFT [13]. There are some previous attempts at naively applying such iterative algorithms as a post-processing step on the magnitudes produced by deep learning based speech enhancement and separation [18,19,20,3].…”

Section: Introductionmentioning

confidence: 99%

End-to-End Speech Separation with Unfolded Iterative Phase Reconstruction

Wang¹,

Roux²,

Wang³

et al. 2018

Interspeech 2018

117

116

View full text Add to dashboard Cite

This paper proposes an end-to-end approach for single-channel speaker-independent multi-speaker speech separation, where time-frequency (T-F) masking, the short-time Fourier transform (STFT), and its inverse are represented as layers within a deep network. Previous approaches, rather than computing a loss on the reconstructed signal, used a surrogate loss based on the target STFT magnitudes. This ignores reconstruction error introduced by phase inconsistency. In our approach, the loss function is directly defined on the reconstructed signals, which are optimized for best separation. In addition, we train through unfolded iterations of a phase reconstruction algorithm, represented as a series of STFT and inverse STFT layers. While mask values are typically limited to lie between zero and one for approaches using the mixture phase for reconstruction, this limitation is less relevant if the estimated magnitudes are to be used together with phase reconstruction. We thus propose several novel activation functions for the output layer of the T-F masking, to allow mask values beyond one. On the publiclyavailable wsj0-2mix dataset, our approach achieves state-ofthe-art 12.6 dB scale-invariant signal-to-distortion ratio (SI-SDR) and 13.1 dB SDR, revealing new possibilities for deep learning based phase reconstruction and representing a fundamental progress towards solving the notoriously-hard cocktail party problem.

show abstract

Section: Introductionmentioning

confidence: 99%

End-to-End Speech Separation with Unfolded Iterative Phase Reconstruction

Wang¹,

Roux²,

Wang³

et al. 2018

Interspeech 2018

117

116

View full text Add to dashboard Cite

show abstract

“…This method is based on binary masking of sources in the MDCT domain, while it is known [16] that oracle bounds of binary masking-based methods are lower than those of Wiener filter-based methods [14], [40]. Another conventional ISS method that is suitable for single channel mixtures is the ISS using iterative reconstruction (ISSIR) by Sturmel and Daudet [41] (see also [42]). ISSIR permits to benefit from phase consistency constraints in the case of STFT representations to reach better performance than Wiener filtering in the case of mono mixtures.…”

Section: A State Of the Art Methodsmentioning

confidence: 99%

Coding-Based Informed Source Separation: Nonnegative Tensor Factorization Approach

Ozerov

Liutkus

Badeau

et al. 2013

IEEE Trans. Audio Speech Lang. Process.

View full text Add to dashboard Cite

Abstract-Informed source separation (ISS) aims at reliably recovering sources from a mixture. To this purpose, it relies on the assumption that the original sources are available during an encoding stage. Given both sources and mixture, a sideinformation may be computed and transmitted along with the mixture, whereas the original sources are not available any longer. During a decoding stage, both mixture and side-information are processed to recover the sources. ISS is motivated by a number of specific applications including active listening and remixing of music, karaoke, audio gaming, etc. Most ISS techniques proposed so far rely on a source separation strategy and cannot achieve better results than oracle estimators. In this study, we introduce Coding-based ISS (CISS) and draw the connection between ISS and source coding. CISS amounts to encode the sources using not only a model as in source coding but also the observation of the mixture. This strategy has several advantages over conventional ISS methods. First, it can reach any quality, provided sufficient bandwidth is available as in source coding. Second, it makes use of the mixture in order to reduce the bitrate required to transmit the sources, as in classical ISS. Furthermore, we introduce Nonnegative Tensor Factorization as a very efficient model for CISS and report rate-distortion results that strongly outperform the state of the art.

show abstract

“…One approach to phase estimation is to promote consistency [18,19], where it modifies the mixture phase depending on the results of the estimated magnitude such that the modified phase satisfies consistency. Some recent works [20][21][22] attempted to combine Wiener filtering with consistency-based techniques. The extension of the above approach incorporating sinusoid models has shown promising results [23].…”

Section: Introductionmentioning

confidence: 99%

PhaseNet: Discretized Phase Modeling with Deep Neural Networks for Audio Source Separation

et al. 2018

View full text Add to dashboard Cite

Previous research on audio source separation based on deep neural networks (DNNs) mainly focuses on estimating the magnitude spectrum of target sources and typically, phase of the mixture signal is combined with the estimated magnitude spectra in an ad-hoc way. Although recovering target phase is assumed to be important for the improvement of separation quality, it can be difficult to handle the periodic nature of the phase with the regression approach. Unwrapping phase is one way to eliminate the phase discontinuity, however, it increases the range of value along with the times of unwrapping, making it difficult for DNNs to model. To overcome this difficulty, we propose to treat the phase estimation problem as a classification problem by discretizing phase values and assigning class indices to them. Experimental results show that our classificationbased approach 1) successfully recovers the phase of the target source in the discretized domain, 2) improves signal-todistortion ratio (SDR) over the regression-based approach in both speech enhancement task and music source separation (MSS) task, and 3) outperforms state-of-the-art MSS.

show abstract

Informed Source Separation Using Iterative Reconstruction

Cited by 24 publications

References 15 publications

End-to-End Speech Separation with Unfolded Iterative Phase Reconstruction

End-to-End Speech Separation with Unfolded Iterative Phase Reconstruction

Coding-Based Informed Source Separation: Nonnegative Tensor Factorization Approach

PhaseNet: Discretized Phase Modeling with Deep Neural Networks for Audio Source Separation

Contact Info

Product

Resources

About