End-To-End Optimized Speech Coding with Deep Neural Networks

Abstract

Modern compression algorithms are often the result of laborious domain-specific research; industry standards such as MP3, JPEG, and AMR-WB took years to develop and were largely hand-designed. We present a deep neural network model which optimizes all the steps of a wideband speech coding pipeline (compression, quantization, entropy coding, and decompression) end-to-end directly from raw speech data -no manual feature engineering necessary, and it trains in hours. In testing, our DNN-based coder performs on par with the AMR-WB standard at a variety of bitrates (∼9kbps up to ∼24kbps). It also runs in realtime on a 3.8GhZ Intel CPU.

Efficient and Scalable Neural Residual Waveform Coding with Collaborative Quantization

“…A 1D-CNN architecture on the time-domain samples serves the desired lightweight autoencoder (AE) for end-to-end speech coding, where the model complexity is a major concern [19,18]. As shown in Table 1, the encoder part consists of four bottleneck ResNet stages [20], a downsampling convolutional layer to halve the feature map size in the middle, and then a channel compression layer to create a real-valued code vector of 256 dimensions.…”

“…To compress speech signals, a core component of this AE is the trainable quantizer which learns a discrete representation of the code layer in the AE. Out of the recent neural network-compatible quantization schemes, such as VQ-VAE [21] and soft-to-hard quantization [22], we focus on soft-to-hard quantization, namely softmax quantization as in the other end-to-end speech coding AEs [19,18]. Given an input frame x ∈ R S of S samples, the output from the encoder is h = FEnc(x), each is a 16-bit floating-point value.…”

Unified Signal Compression Using Generative Adversarial Networks

“…Traditional speech codec methods such as CELP [6], Opus [7], and adaptive multirate wideband (AMR-WB) [8] commonly employ hand-engineered encoder-decoder pipelines relying on manually/mathematically crafted audio representation features and/or signal prediction models. Recent DNN based approaches including [9] demonstrate the feasibility to train an end-to-end speech codec that exhibits performance comparable to a hand-crafted AMR-WB codec at 9-24 kbps. [10] uses deep spiking neural networks to realizes a low bit rate speech codec.…”

A Dual-Staged Context Aggregation Method towards Efficient End-to-End Speech Enhancement

“…The TF domain regularizer compensates the end-to-end DNN that would only operate in time domain, otherwise. Empirically, it is shown to achieve better perceptual quality, as proposed in [20].…”