Two Heads are Better Than One: A Two-Stage Complex Spectral Mapping Approach for Monaural Speech Enhancement

Li, Andong; Liu, Wenzhe; Zheng, Chengshi; Fan, Cunhang; Li, Xiaodong

doi:10.1109/taslp.2021.3079813

Cited by 105 publications

(41 citation statements)

References 57 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In Eq. ( 18), the full loss function is a linear combination of the magnitude and RI loss functions, as it is reported that these two terms together improve the speech quality [10], [28], [29].…”

Section: F Loss Functionmentioning

confidence: 99%

“…In this study, we first compare our model with various advanced systems on WSJ0-SI84 + DNS dataset. For fair comparison, we re-implement all the baselines with noncausal setting, namely BiLSTM [38], BiCRN [18], GRN [39], DCN [40], AECNN [41], ConvTasNet [42], BiDCCRN [43], BiGCRN [14] and CTS-Net [10] (Non-causal version), where BiLSTM, BiCRN, and BiGCRN are similar to that of LSTM, CRN, and GCRN, except that all the LSTM layers are replaced by their bidirectional versions.…”

Section: Baselinesmentioning

confidence: 99%

“…For example, SEGAN [9] proposed a generative adversarial network-based SE method, where a denoising generator directly maps the raw waveform of the clean speech from the mixed raw waveform by adversarial training. However, it was reported that the fine-detailed structures of noise and speech components are more separable with T-F representation [10]. Hence, the latter category handles the phase estimation in the complex domain, which can be divided into two main streams, namely maskingbased and mapping-based.…”

Section: Introductionmentioning

confidence: 99%

“…For the first type, a multitude of complex-valued DNN-based algorithms have been proposed to estimate the complex-valued ratio mask (CRM), which can be then applied to real and imaginary (RI) parts of the complex spectrum, so as to recover the magnitude and phase simultaneously [11]- [13]. The second type has leveraged complex spectral mapping networks to directly predict the RI components of the clean complex spectrum [10], [14]. For example, in [14], real-valued convolutional recurrent networks (CRN) were leveraged to directly map the RI components of target speech, where the enhanced RI components were decoded by two decoders respectively.…”

Section: Introductionmentioning

confidence: 99%

“…For example, in [14], real-valued convolutional recurrent networks (CRN) were leveraged to directly map the RI components of target speech, where the enhanced RI components were decoded by two decoders respectively. More recently, a handful of multi-stage decoupling-style methods have thrived in the SE area and were demonstrated to achieve a remarkable performance [10], [15], [16]. Instead of packing the mapping process into only one black box in the previous single-stage paradigm, these multi-stage methods decoupled the original complex spectrum estimation into optimizing magnitude and phase stage by stage, and alleviated the implicit compensation effect between two targets [17].…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

DBT-Net: Dual-branch federative magnitude and phase estimation with attention-in-attention transformer for monaural speech enhancement

Yu¹,

Li²,

Wang³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

The decoupling-style concept begins to ignite in the speech enhancement area, which decouples the original complex spectrum estimation task into multiple easier sub-tasks (i.e., magnitude and phase), resulting in better performance and easier interpretability. In this paper, we propose a dual-branch federative magnitude and phase estimation framework, dubbed DBT-Net, for monaural speech enhancement, which aims at recovering the coarse-and fine-grained regions of the overall spectrum in parallel. From the complementary perspective, the magnitude estimation branch is designed to filter out dominant noise components in the magnitude domain, while the complex spectrum purification branch is elaborately designed to inpaint the missing spectral details and implicitly estimate the phase information in the complex domain. To facilitate the information flow between each branch, interaction modules are introduced to leverage features learned from one branch, so as to suppress the undesired parts and recover the missing components of the other branch. Instead of adopting the conventional RNNs and temporal convolutional networks for sequence modeling, we propose a novel attention-in-attention transformer-based network within each branch for better feature learning. More specially, it is composed of several adaptive spectro-temporal attention transformerbased modules and an adaptive hierarchical attention module, aiming to capture long-term time-frequency dependencies and further aggregate intermediate hierarchical contextual information. Comprehensive evaluations on the WSJ0-SI84 + DNS-Challenge and VoiceBank + DEMAND dataset demonstrate that the proposed approach consistently outperforms previous advanced systems and yields state-of-the-art performance in terms of speech quality and intelligibility.

show abstract

Section: F Loss Functionmentioning

confidence: 99%