Generalized Difference Coder: A Novel Conditional Autoencoder Structure for Video Compression

Abstract

Motion compensated inter prediction is a common component of all video coders. The concept was established in traditional hybrid coding and successfully transferred to learning-based video compression. To compress the residual signal after prediction, usually the difference of the two signals is compressed using a standard autoencoder. However, information theory tells us that a general conditional coder is more efficient. In this paper, we provide a solid foundation based on information theory and Shannon entropy to show the potentials but also the limits of conditional coding. Building on those results, we then propose the generalized difference coder, a special case of a conditional coder designed to avoid limiting bottlenecks. With this coder, we are able to achieve average rate savings of 27.8% compared to a standard autoencoder, by only adding a moderate complexity overhead of less than 7%.

CANF-VC: Conditional Augmented Normalizing Flows for Video Compression

“…The idea was extended in [22] for conditional motion coding, which encodes motion latents in an implicit, one-stage manner. However, Fabian et al [6] show that these conditional VAE-based approaches [21,22] may suffer from the bottleneck issue; that is, the latent representation of x c produced by a neural network for conditional decoding may not capture all the information of x c , which serves as a condition for encoding x t . Such information loss and asymmetry can harm the efficiency of conditional coding.…”

“…As compared with DCVC [23], CANF-VC additionally features conditional motion coding. Although conditional motion coding also appears in [22], their VAE-based approach does not explicitly estimate a flow map prior to conditional coding, and may suffer from the bottleneck issue [6] (Section 2.2). In contrast, CANF-VC takes an explicit approach and avoids the bottleneck issue by using the same x c symmetrically in the encoder and the decoder due to its invertible property.…”