Low-latency Deep Clustering for Speech Separation

Wang, Shanshan; Naithani, Gaurav; Virtanen, Tuomas

doi:10.1109/icassp.2019.8683437

Cited by 12 publications

(13 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In TF spectrum based speech separation with DNNs, the training targets for supervised learning are usually in the form of a TF representation, e.g., TF masks or affinity matrices (for deep clustering [5], [18]). The STFT is a popular choice where the choice of the window length is important.…”

Section: Proposed Methodsmentioning

confidence: 99%

“…In order to show the independence of our proposed approach from the type of models and datasets used, we evaluate it on two tasks: speaker-independent separation with an online DC model [18], and speaker-dependent separation with mask inference (MI) network that directly predicts masks. We evaluate the former and latter on two-speaker mixtures from Wall Street Journal (WSJ0) [19] and Danish HINT [20,21] databases, respectively.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Deep Neural Network Based Low-Latency Speech Separation with Asymmetric Analysis-Synthesis Window Pair

Wang¹,

Naithani²,

Politis³

et al. 2021

2021 29th European Signal Processing Conference (EUSIPCO)

Self Cite

View full text Add to dashboard Cite

Time-frequency masking or spectrum prediction computed via short symmetric windows are commonly used in lowlatency deep neural network (DNN) based source separation. In this paper, we propose the usage of an asymmetric analysis-synthesis window pair which allows for training with targets with better frequency resolution, while retaining the low-latency during inference suitable for real-time speech enhancement or assisted hearing applications. In order to assess our approach across various model types and datasets, we evaluate it with both speaker-independent deep clustering (DC) model and a speaker-dependent mask inference (MI) model. We report an improvement in separation performance of up to 1.5 dB in terms of source-to-distortion ratio (SDR) while maintaining an algorithmic latency of 8 ms.

show abstract

Section: Proposed Methodsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Deep Neural Network Based Low-Latency Speech Separation with Asymmetric Analysis-Synthesis Window Pair

Wang¹,

Naithani²,

Politis³

et al. 2021

2021 29th European Signal Processing Conference (EUSIPCO)

Self Cite

View full text Add to dashboard Cite

show abstract

“…Many techniques reported consisted of multiple stages separately optimized under different criteria, such as signal representation and embedding [15]. Some embedding clustering methods add phase information on multi-channel, and other research concerns the low-delay of deep clustering approaches [16], [17]. Orthogonal deep clustering improves the separation performance of the model by adding an orthogonal constraint penalty term of the objective function to reduce the correlation between the embedded expression [18].…”

Section: Related Work a Speech Separation Based On Deep Clusteringmentioning

confidence: 99%

Graph Convolution-Based Deep Clustering for Speech Separation

Qin

Jiang

et al. 2020

IEEE Access

View full text Add to dashboard Cite

Deep clustering is a promising technique for speech separation that is crucial to speech communication, acoustic target detection, acoustic enhancement and speech recognition. In the study of monophonic speech separation, the problem is that the decrease in separation and generalization performance of the model in the case of reducing the variety of the training data set. In this paper, we propose a comprehensive deep clustering framework that construction the structural speech data based on GCN, named graph deep clustering (GDC) to further improve the separation performance of the separation model. In particular, embedding features are transformed into graph-structured data, and the speech separation mask is achieved by clustering these graph-structured data. Graph structural information aggregates nodes within a class, which makes feature representations conducive to clustering. Experimental results demonstrate that the proposed scheme can improve the clustering performance. The SDR of the separated speech is improved by about 1.2 dB, and the clustering accuracy is improved by 15%. We also use the perceptually motivated objective measures for the evaluation of audio source separation to score the speech quality. The target speech quality and the overall perceptual score are improved by 10.7% compared with other speech separation algorithms. INDEX TERMS Construction of graph-structured data, deep clustering, graph convolutional filter, speech separation.

show abstract

“…However, the MISI algorithm has been introduced in a heuristic fashion, therefore there is currently no proof that it converges. Besides, while several recent works addressed the problem of low-latency magnitude estimation [20], [21], [22], the MISI algorithm operates offline, as it computes the whole STFT and its inverse at each iteration. This makes it impracticable for real-time applications such as hearing aids.…”

Section: Introduction Audio Source Separationmentioning

confidence: 99%

Online Spectrogram Inversion for Low-Latency Audio Source Separation

Magron

Virtanen

2020

IEEE Signal Process. Lett.

Self Cite

View full text Add to dashboard Cite

Audio source separation is usually achieved by estimating the short-time Fourier transform (STFT) magnitude of each source, and then applying a spectrogram inversion algorithm to retrieve time-domain signals. In particular, the multiple input spectrogram inversion (MISI) algorithm has been exploited successfully in several recent works. However, this algorithm suffers from two drawbacks, which we address in this paper. First, it has originally been introduced in a heuristic fashion: we propose here a rigorous optimization framework in which MISI is derived, thus proving the convergence of this algorithm. Besides, while MISI operates offline, we propose here an online version of MISI called oMISI, which is suitable for low-latency source separation, an important requirement for e.g., hearing aids applications. oMISI also allows one to use alternative phase initialization schemes exploiting the temporal structure of audio signals. Experiments conducted on a speech separation task show that oMISI performs as well as its offline counterpart, thus demonstrating its potential for real-time source separation.

show abstract

Low-latency Deep Clustering for Speech Separation

Cited by 12 publications

References 22 publications

Deep Neural Network Based Low-Latency Speech Separation with Asymmetric Analysis-Synthesis Window Pair

Deep Neural Network Based Low-Latency Speech Separation with Asymmetric Analysis-Synthesis Window Pair

Graph Convolution-Based Deep Clustering for Speech Separation

Online Spectrogram Inversion for Low-Latency Audio Source Separation

Contact Info

Product

Resources

About