Environmental Attention-Guided Branchy Neural Network for Speech Enhancement

Abstract

The performance of speech enhancement algorithms can be further improved by considering the application scenarios of speech products. In this paper, we propose an attention-based branchy neural network framework by incorporating the prior environmental information for noise reduction. In the whole denoising framework, first, an environment classification network is trained to distinguish the noise type of each noisy speech frame. Guided by this classification network, the denoising network gradually learns respective noise reduction abilities in different branches. Unlike most deep neural network (DNN)-based methods, which learn speech reconstruction capabilities with a common neural structure from all training noises, the proposed branchy model obtains greater performance benefits from the specially trained branches of prior known noise interference types. Experimental results show that the proposed branchy DNN model not only preserved better enhanced speech quality and intelligibility in seen noisy environments, but also obtained good generalization in unseen noisy environments.for the performance improvement of DNN-based speech denoising. Generative adversarial networks (GAN) [22] have been another new solution to train the DNN model to learn noise suppression abilities. These methods aim to train a common DNN denoising model for all kinds of noise interference in different application scenarios.However, most speech products have their specific application scenarios. When the application scenario is determined, the types of noise interference are known. In different noise environments, the optimal parameters of speech enhancement algorithms are different. As investigated in [23], this prior environmental information was helpful to further improve the speech enhancement methods to achieve better noise suppression effect in specific noise environments. Therefore, some researchers started to design the speech enhancement algorithms by incorporating the prior noise information. In [24][25][26], the noise classification module was integrated into the Wiener filter and some statistical model-based speech estimators. Guided by the noise classification module for optimal parameter selection, the performance of traditional methods was further improved in prior known noisy environments. A similar idea was applied to the DNN-based denoising algorithms [27][28][29]. In the recent studies of [28,29], under the guidance of the noise classification unit, several independent DNN models were trained to give full play to their respective noise reduction capabilities in different noise environments. Although this "divide and conquer" strategy effectively improves the noise reduction performance, it also increases the storage burden and reduces the complementarity between different noises.In this work, we proposed a novel branchy neural network (BNN) framework to improve the storage burden and noise complementarity problem of separate training. There are two key modules working together in our proposed framework, a classification...

Singular Spectrum Analysis for Source Separation in Drone-Based Audio Recording

“…Neural network-based approaches can achieve state-ofthe-art results in application domains with great amounts of data. RNNoise [20], put forward by Zhang et al [21], and the Wang et al method [22] are both great alternatives in the voice denoising field. However, they are not as performant in domains with much less data.…”

“…Three kinds of sounds were recorded: sounds emitted by the drone flying with different motor speeds, sounds emitted by a source, and a mix of the two. In the validation process, we created synthetic mixes using the expression (21). Once such example is portrayed in Fig.…”

“…The mixes used to perform this experiment have been generated synthetically, using drone sounds and source sounds from the dataset. The mixing system is the one described in (21), where d is the relative power of the source with respect to the drone in decibels.…”

“…Content may change prior to final publication. • s source (21) All the possible combinations of the parameters have been tested. In all of them, the error has been measured using the root-mean-square error normalized by the mean of the reference signal (NRMSE).…”

“…The relative power took 50 equispaced values between −5db and 5db, as in the experiment above. Again, in this experiment, drone and source sounds were mixed using (21). A total of 1250 executions were performed and the error was once again estimated using NRMSE.…”