PostGAN: A GAN-Based Post-Processor to Enhance the Quality of Coded Speech

Korse, Srikanth; Pia, Nicola; Gupta, Krishnendu; Fuchs, Guillaume

doi:10.1109/icassp43922.2022.9747733

Cited by 4 publications

(2 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As existing predictive methods are only applied as postfilters, they can only reduce the introduced artefacts in the signal but cannot restore areas of speech that are completely attenuated during lossy encoding. Generative models including auto-regressive models [18] and Generative Adversarial Networks (GAN) [17], [19] are other attempts for coded speech enhancement, which can overcome this limitation. Thus, the bitrate and codec-informed combination of post-filtering with generative models is an interesting future direction to explore.…”

Section: B Future Extensionsmentioning

confidence: 99%

Bitrate-Informed Coded Speech Enhancement Model

Zhao,

Madhu

2024

2024 International Conference on Electronics, Information, and Communication (ICEIC)

View full text Add to dashboard Cite

Enhancing speech processed by lossy codecs can significantly improve the resultant signal quality, providing a richer listening experience while reducing listening fatigue. Since there are a multitude of codecs, each supporting several bitrates, deep-learning-based solutions typically train networks in a codecspecific manner and use multi-condition training for each codecspecific network, to improve generalizability for the different bit-rates. In contrast, we propose a bitrate-informed model for improving the inter-bitrate generalizability of the model for coded speech enhancement. The well-known Convolutional Recurrent U-Net Speech Enhancement (CRUSE) encoder-decoder model is selected for the enhancement, however, we propose modifying only the initial few layers of the encoder, to introduce bitrate dependency. The rest of the network is shared for all bitrates. Evaluation is carried out on two contemporary codecs: the Bluetooth low complexity communication codec plus (LC3plus) and the 3GPP adaptive multi-rate wideband (AMR-WB) codec. The experimental study shows that using bitrate-informed layers improves generalisation capability. More importantly, this only causes a small increase (< 1%) in model footprint and no increase in the computational cost. Further, to provide better insights into where such bitrate-informed layers can be useful, we propose using the histogram of the ideal training target masks. The radically different histograms at the different bitrates for LC3plus codec indicate a stronger benefit with the bitrateinformed model -which is also seen in the instrumental metrics. This paper lays the foundation for further work on bitrate and codec-informed models -allowing for the development of a single, universal model for coded speech enhancement.

show abstract

Section: B Future Extensionsmentioning

confidence: 99%

Bitrate-Informed Coded Speech Enhancement Model

Zhao,

Madhu

2024

2024 International Conference on Electronics, Information, and Communication (ICEIC)

View full text Add to dashboard Cite

show abstract

“…It adopted a lightweight Transformer [11] for additional coding gain but at the cost of increased algorithmic delay. Recently reported NSCs introduced various advantages in low bitrates, such as T-F codec [12] for low latency, DAC [13] and Post-GAN [14] for high sound quality, etc., but they rarely targeted the efficiency goal. Likewise, there is a tradeoff between the model complexity and coding gain in the NSC literature, which we tackle in this paper by proposing personalized neural speech coding (PNSC).…”

Section: Introductionmentioning

confidence: 99%

Personalized Neural Speech Codec

Jang,

Yang,

Lim

et al. 2024

ICASSP 2024 - 2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

In this paper, we propose a personalized neural speech codec, envisioning that personalization can reduce the model complexity or improve perceptual speech quality. Despite the common usage of speech codecs where only a single talker is involved on each side of the communication, personalizing a codec for the specific user has rarely been explored in the literature. First, we assume speakers can be grouped into smaller subsets based on their perceptual similarity. Then, we also postulate that a group-specific codec can focus on the group's speech characteristics to improve its perceptual quality and computational efficiency. To this end, we first develop a Siamese network that learns the speaker embeddings from the LibriSpeech dataset, which are then grouped into underlying speaker clusters. Finally, we retrain the LPCNet-based speech codec baselines on each of the speaker clusters. Subjective listening tests show that the proposed personalization scheme introduces model compression while maintaining speech quality. In other words, with the same model complexity, personalized codecs produce better speech quality.

show abstract