Invertible DNN-Based Nonlinear Time-Frequency Transform for Speech Enhancement

Takeuchi, Daisuke; Yatabe, Kohei; Oikawa, Yasuhiro; Harada, Noboru

doi:10.1109/icassp40776.2020.9053723

Cited by 9 publications

(5 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For image processing, an invertible DNN named i-Revnet has been developed [29] and has recently been used as a trainable time-frequency transform of a DNN-based speech enhancement system [30]. i-Revnet alternately performs the squeezing operation and the application of nonlinear functions to only half of each output of the squeezing operation.…”

Section: A Conventional Ds Layersmentioning

confidence: 99%

Time-Domain Audio Source Separation With Neural Networks Based on Multiresolution Analysis

Nakamura

Kozuka

Saruwatari

2021

IEEE/ACM Trans. Audio Speech Lang. Process.

View full text Add to dashboard Cite

We propose a time-domain audio source separation method based on multiresolution analysis, which we call multiresolution deep layered analysis (MRDLA). The MRDLA model is based on one of the state-of-the-art time-domain deep neural networks (DNNs), Wave-U-Net, which successively downsamples features and up-samples them to have the original time resolution. From the signal processing viewpoint, we found that the down-sampling (DS) layers of Wave-U-Net cause aliasing and may discard information useful for source separation because they are implemented with decimation. These two problems are due to the decimation; thus, to achieve a more reliable source separation method, we should design DS layers capable of simultaneously overcoming these problems. With this motivation, focusing on the fact that the successive DS architecture of Wave-U-Net resembles that of multiresolution analysis, we develop DS layers based on discrete wavelet transforms (DWTs), which we call the DWT layers, because the DWTs have anti-aliasing filters and the perfect reconstruction property. We further extend the DWT layers such that their wavelet basis functions can be trained together with the other DNN components while maintaining the perfect reconstruction property. Since a straightforward trainable extension of the DWT layers does not guarantee the existence of anti-aliasing filters, we derive constraints for this guarantee in addition to the perfect reconstruction property. Through music source separation experiments including subjective evaluations, we show the efficacy of the proposed methods and the importance of simultaneously considering both the anti-aliasing filters and the perfect reconstruction property.

show abstract

Section: A Conventional Ds Layersmentioning

confidence: 99%

Time-Domain Audio Source Separation With Neural Networks Based on Multiresolution Analysis

Nakamura

Kozuka

Saruwatari

2021

IEEE/ACM Trans. Audio Speech Lang. Process.

View full text Add to dashboard Cite

show abstract

“…An important requirement in DNN-based speech enhancement and separation is generalization that means working for any speaker. To achieve this, in speech enhancement, several studies train a global M using many speech samples spoken by many speakers [3][4][5][6][7][8][9][10][11][12][13][14]. Unfortunately, in speech separation, generalization cannot be achieved solely using a large scale training dataset because there is no way of knowing which signal in the speech-mixture is the target.…”

Section: Auxiliary Speaker-aware Feature For Speech Separationmentioning

confidence: 99%

“…Generalization is an important requirement in DNN-based speech enhancement to enable enhancing unknown speakers' speech. To achieve this, several previous studies train a speaker independent DNN using many speech samples spoken by many speakers [3][4][5][6][7][8][9][10][11][12][13][14]. Meanwhile, in other speech applications, model specialization to the target speaker has succeeded [15,16].…”

Section: Introductionmentioning

confidence: 99%

Speech Enhancement Using Self-Adaptation and Multi-Head Self-Attention

Koizumi

Yatabe

Delcroix

et al. 2020

ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

Self Cite

105

View full text Add to dashboard Cite

This paper investigates a self-adaptation method for speech enhancement using auxiliary speaker-aware features; we extract a speaker representation used for adaptation directly from the test utterance. Conventional studies of deep neural network (DNN)-based speech enhancement mainly focus on building a speaker independent model. Meanwhile, in speech applications including speech recognition and synthesis, it is known that model adaptation to the target speaker improves the accuracy. Our research question is whether a DNN for speech enhancement can be adopted to unknown speakers without any auxiliary guidance signal in test-phase. To achieve this, we adopt multi-task learning of speech enhancement and speaker identification, and use the output of the final hidden layer of speaker identification branch as an auxiliary feature. In addition, we use multi-head self-attention for capturing long-term dependencies in the speech and noise. Experimental results on a public dataset show that our strategy achieves the state-of-the-art performance and also outperform conventional methods in terms of subjective quality.

show abstract

“…This setup is a standard architecture in DNNbased speech enhancement [6]. The input of the DNN was logamplitude spectrogram of the observed signal x whose size was F × K. The kernel size, stride, and padding of both 2-D CNNs were (5,15), (1,1) and (2,7), respectively. The number of output channels of the first and second 2-D CNNs were 30 and 60, respectively.…”

Section: Experimental Setupsmentioning

confidence: 99%

“…Over the last decade, the use of deep neural network (DNN) for speech enhancement has substantially advanced the state-ofthe-art performance [3][4][5][6][7][8][9][10][11][12][13][14][15]. The popular strategy is to estimate a time-frequency (T-F) mask by a DNN and apply it in the short-time Fourier transform (STFT)-domain [3], where the enhanced signal is obtained by the inverse STFT.…”

Section: Introductionmentioning

confidence: 99%

Stable Training of Dnn for Speech Enhancement Based on Perceptually-Motivated Black-Box Cost Function

Kawanaka

Miyazaki

Yatabe

2020

ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

Self Cite

View full text Add to dashboard Cite

Improving subjective sound quality of enhanced signals is one of the most important missions in speech enhancement. For evaluating the subjective quality, several methods related to perceptuallymotivated objective sound quality assessment (OSQA) have been proposed such as PESQ (perceptual evaluation of speech quality). However, direct use of such measures for training deep neural network (DNN) is not allowed in most cases because popular OSQAs are non-differentiable with respect to DNN parameters. Therefore, the previous study has proposed to approximate the score of OS-QAs by an auxiliary DNN so that its gradient can be used for training the primary DNN. One problem with this approach is instability of the training caused by the approximation error of the score. To overcome this problem, we propose to use stabilization techniques borrowed from reinforcement learning. The experiments, aimed to increase the score of PESQ as an example, show that the proposed method (i) can stably train a DNN to increase PESQ, (ii) achieved the state-of-the-art PESQ score on a public dataset, and (iii) resulted in better sound quality than conventional methods based on subjective evaluation.Index Terms-Speech enhancement, sound quality assessment, perceptual evaluation of speech quality, function approximation. A q 5 i c q i d n B a Z i l Ka c X u q c 6 P d M D v d L n n 7 2 q Q Y + G l w p n t a k V T m H 0 f G r v 4 1 + V y d n H 0 b e q r W c f h 1 g K v O r s 3 Q m Y x i m 0 p r 5 8 e l H f W 9 6 N V 2 f p h t 7 Y / z X V 6 I l P Y J X f t d s d s X v Z x o / K X v j G e E D y 7 3 G 0 g t R 8 Q q a E v L M Q W 6 V w V H 2 Y x g z m e B 6 L W M U m t p H k / m V c 4 R Z 3 0 q S 0 I q 1 L G A q 5 i c q i d n B a Z i l K A g o q 1 A D k E M Y W z b k U f k c A A b G k o w I W D B Z 2 x A g c d P F j I I D n N 5 V J l z G e n B v s A Z B l h b 4 i r B F Q q z x / w t 8 i o b s h a v G z 2 9 Q K 3 x X w x + X V Z G E a c X u q c 6 P d M D v d L n n 7 2 q Q Y + G l w p n t a k V T m H 0 f G r v 4 1 + V y d n H 0 b e q r W c f h 1 g K v O r s 3 Q m Y x i m 0 p r 5 8 e l H f W 9 6 N V 2 f p h t 7 Y / z X V 6 I l P Y J X f t d s d s X v Z x o / K X v j G e E D y 7 3 G 0 g t R 8 Q q a E v L M Q W 6 V w V H 2 Y x g z m e B 6 L W M U m t p H k / m V c 4 R Z 3 0 q S 0 I q 1 L G A q 5 i c q i d n B a Z i l K A g o q 1 A D k E M Y W z b k U f k c A A b G k o w I W D B Z 2 x A g c d P F j I I D n N 5 V J l z G e n B v s A Z B l h b 4 i r B F Q q z x / w t 8 i o b s h a v G z 2 9 Q K 3 x X w x + X V Z G E a c X u q c 6 P d M D v d L n n 7 2 q Q Y + G l w p n t a k V T m H 0 f G r v 4 1 + V y d n H 0 b e q r W c f h 1 g K v O r s 3 Q m Y x i m 0 p r 5 8 e l H f W 9 6 N V 2 f p h t 7 Y / z X V 6 I l P Y J X f t d s d s X v Z x o / K X v j G e E D y 7 3 G 0 g t R 8 Q q a E v L M Q W 6 V w V H 2 Y x g z m e B 6 L W M U m t p H k / m V c 4 R Z 3 0 q S 0 I q 1 L G A q 5 i c q i d n B a Z i l K A g o q 1 A D k E M Y W z b k U f k c A A b G k o w I W D B Z 2 x A g c d P F j I I D n N 5 V J l z G e n B v s A Z B l h b 4 i r B F Q q z x / w t 8 i o b s h...

show abstract

Invertible DNN-Based Nonlinear Time-Frequency Transform for Speech Enhancement

Cited by 9 publications

References 30 publications

Time-Domain Audio Source Separation With Neural Networks Based on Multiresolution Analysis

Time-Domain Audio Source Separation With Neural Networks Based on Multiresolution Analysis

Speech Enhancement Using Self-Adaptation and Multi-Head Self-Attention

Stable Training of Dnn for Speech Enhancement Based on Perceptually-Motivated Black-Box Cost Function

Contact Info

Product

Resources

About