Time-Frequency Mask-based Speech Enhancement using Convolutional Generative Adversarial Network

Shah, Neil; Patil, Hemant A.; Soni, Meet H.

doi:10.23919/apsipa.2018.8659692

Cited by 35 publications

(22 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The phase of noisy speech is utilized along with the enhanced speech magnitude to reconstruct the time-domain signal with inverse short-time Fourier transform (iSTFT). Despite some inspiring results achieved [1][2][3][4], there are still two main limitations in T-F domain methods. First, Fourier transform operation is an additional overhead which hinders fast speech denoising.…”

Section: Introductionmentioning

confidence: 99%

TSTNN: Two-Stage Transformer Based Neural Network for Speech Enhancement in the Time Domain

Wang

Zhu

2021

ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

103

View full text Add to dashboard Cite

In this paper, we propose a transformer-based architecture, called two-stage transformer neural network (TSTNN) for end-to-end speech denoising in the time domain. The proposed model is composed of an encoder, a two-stage transformer module (TSTM), a masking module and a decoder. The encoder maps input noisy speech into feature representation. The TSTM exploits four stacked two-stage transformer blocks to efficiently extract local and global information from the encoder output stage by stage. The masking module creates a mask which will be multiplied with the encoder output. Finally, the decoder uses the masked encoder feature to reconstruct the enhanced speech. Experimental results on the benchmark dataset show that the TSTNN outperforms most state-of-theart models in time or frequency domain while having significantly lower model complexity.

show abstract

Section: Introductionmentioning

confidence: 99%

TSTNN: Two-Stage Transformer Based Neural Network for Speech Enhancement in the Time Domain

Wang

Zhu

2021

ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

103

View full text Add to dashboard Cite

show abstract

“…Assessing the quality of audio signals is an important consideration in many audio and multimedia applications, such as speech recognition, high-quality music recording, and machine fault detection. By now, there have been some studies in audio quality assessment [1,2,3,4]. Fu et al [2] developed a non-intrusive speech quality evaluation model to predict PESQ scores using a BLSTM model.…”

Section: Introductionmentioning

confidence: 99%

“…Fu et al [2] developed a non-intrusive speech quality evaluation model to predict PESQ scores using a BLSTM model. Avila et al, [3] investigated the applicability of three neural network-based approaches for non-intrusive audio quality assessment based on mean opinion score(MOS) estimation [5]. These previous studies mainly focused on quality estimations of audio recordings by predicting a quality score.…”

Section: Introductionmentioning

confidence: 99%

Improving Audio Anomalies Recognition Using Temporal Convolutional Attention Networks

Huang

Hain

2021

ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

Anomalous audio in speech recordings is often caused by speaker voice distortion, external noise, or even electric interferences. These obstacles have become a serious problem in some fields, such as high-quality dubbing and speech processing. In this paper, a novel approach using a temporal convolutional attention network (TCAN) is proposed to tackle this problem. The use of temporal conventional network (TCN) can capture long range patterns using a hierarchy of temporal convolutional filters. To enhance the ability to tackle audio anomalies in different acoustic conditions, an attention mechanism is used in TCN, where a self-attention block is added after each temporal convolutional layer. This aims to highlight the target related features and mitigate the interferences from irrelevant information. To evaluate the performance of the proposed model, audio recordings are collected from the TIMIT dataset, and are then changed by adding five different types of audio distortions: gaussian noise, magnitude drift, random dropout, reduction of temporal resolution, and time warping. Distortions are mixed at different signal-to-noise ratios (SNRs) (5dB, 10dB, 15dB, 20dB, 25dB, 30dB). The experimental results show that the use of proposed model can yield better classification performances than some strong baseline methods, such as the LSTM and TCN based models, by approximate 3∼ 10% relative improvements.

show abstract

“…Many existing DNN-based speech denoising algorithms are implemented in the time-frequency (T-F) domain, where only the spectral magnitude is enhanced and the noisy phase is used for speech reconstruction. Some of those works such as [1]- [4], have achieved inspiring results. However, there are two main limitations in the T-F domain methods.…”

Section: Introductionmentioning

confidence: 99%

CAUNet: Context-Aware U-Net for Speech Enhancement in Time Domain

Wang

Zhu

2021

2021 IEEE International Symposium on Circuits and Systems (ISCAS)

View full text Add to dashboard Cite

In this paper, we propose a transformer neural network based U-Net architecture, called context-aware U-Net (CAUNet), for end-to-end speech denoising in time domain. The proposed model adopts the dilated-dense block in both encoder and decoder layers of the U-Net to strengthen feature propagation and enlarge the receptive field of features. It also uses stacked two-stage transformer blocks to efficiently extract local and global contextual information from the encoder output, based on which the enhanced speech is reconstructed at the decoder. Experimental results show that our model outperforms most state-of-the-art methods in time and frequency domains, while it maintains a relatively low model complexity.

show abstract

Time-Frequency Mask-based Speech Enhancement using Convolutional Generative Adversarial Network

Cited by 35 publications

References 24 publications

TSTNN: Two-Stage Transformer Based Neural Network for Speech Enhancement in the Time Domain

TSTNN: Two-Stage Transformer Based Neural Network for Speech Enhancement in the Time Domain

Improving Audio Anomalies Recognition Using Temporal Convolutional Attention Networks

CAUNet: Context-Aware U-Net for Speech Enhancement in Time Domain

Contact Info

Product

Resources

About