Enhancement of Coded Speech Using a Mask-Based Post-Filter

Abstract

The quality of speech codecs deteriorates at low bitrates due to high quantization noise. A post-filter is generally employed to enhance the quality of the coded speech. In this paper, a data-driven postfilter relying on masking in the time-frequency domain is proposed. A fully connected neural network (FCNN), a convolutional encoderdecoder (CED) network and a long short-term memory (LSTM) network are implemeted to estimate a real-valued mask per timefrequency bin. The proposed models were tested on the five lowest operating modes (6.65 kbps-15.85 kbps) of the Adaptive Multi-Rate Wideband codec (AMR-WB). Both objective and subjective evaluations confirm the enhancement of the coded speech and also show the superiority of the mask-based neural network system over a conventional heuristic post-filter used in the standard like ITU-T G.718.

Enhancement of Coded Speech Using Neural Network-Based Side Information

“…The objective measure used in the experiments was the wideband PESQ scores [42], which are designed to mimic the ITU-T Recommendation P.800 Absolute Category Rating (ACR) Mean Opinion Score (MOS) test scores [48]. The average PESQ scores for the decoded signals, outputs of the mask-based post-filter in [17], outputs of the ffDNNbased post-processing without side information, and outputs of the proposed method utilizing side information for the HE-AAC operating at various bitrates are shown in Fig. 3.…”

“…The neural network models for side information extraction and post-processing can be any deep learning model, such as feed-forward DNN (ffDNN), CNN, RNN, and adversarial loss-based models in the time-or frequency-domain [14]- [17]. In this study, we employed the simplest model to confirm if neural network-based side information generation and post-processing are effective.…”

“…It is noted that the current work focused on demonstrating the effectiveness of using neural networkbased side information for coded speech enhancement, and computational complexity was not of primary concern. We compared the performance of the proposed method to that of the decoded signals for the same codec operating at higher bitrates enhanced with ffDNN-based post-processing without any side information and the mask-based post-filter proposed in [17]. The ffDNN structure for post-processing without side information was also 1024-1024-1024 units for three hidden layers, which was the same as that for the post-processing g in the proposed method, except for the input dimension.…”

“…The ffDNN structure for post-processing without side information was also 1024-1024-1024 units for three hidden layers, which was the same as that for the post-processing g in the proposed method, except for the input dimension. The mask-based post-filter [17] had a convolutional encoder-decoder structure that takes the log magnitude spectra for the current and previous frames to estimate the ideal ratio masks bounded by 4 and 2 for HE-AAC and AMR-WB, respectively. The ffDNN-based postprocessing without side information and the mask-based post-filter required approximately 197 WMOPS and 1187 WMOPS, respectively.…”

“…Signal restoration using generative adversarial networks was proposed in [16], by assuming that the generative model is capable of recovering components lost by low bitrate coding. Mask-based post-filter was also investigated by employing a convolutional encoder-decoder [17]. In [18], [19], and [20], decoders of parametric coders were constructed using neural speech synthesizers based on WaveNet [21], SampleRNN [22], and LPCNet [23], respectively.…”