Learning to compress videos without computing motion

Chen, Meixu; Goodall, Todd; Patney, Anjul; Bovik, Alan C.

doi:10.1016/j.image.2022.116633

Cited by 6 publications

(4 citation statements)

References 34 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Figure 1 illustrates the overall architecture of our network, which extends our previous MOVI-Codec [12]. The compression network is comprised of four components: a Displacement Calculation Unit (DCU), a Displacement Compression Network (DCN), a Foveation Generator Unit (FGU), and a Frame Reconstruction Network (FRN).…”

Section: A Frameworkmentioning

confidence: 99%

“…The quantized map in x axis is shown in Figure 4. After a set of n masks M (P ) are generated, we element-wise multiply M (P ) and the encoder output y t to obtain quantized spatially variant (foveated) codes c t which are then subjected to entropy coding and bitrate estimation, using the same procedure as [12].…”

Section: Foveation Generator Unit (Fgu)mentioning

confidence: 99%

“…Lu et al [11] developed a video compression network called DVC, where each component of a traditional hybrid codec is replaced by a deep network. In [12], a deep video compression without motion estimation and compensation is proposed, and the resulting codec is competitive to latest H.266 on high resolution videos against MS-SSIM.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Foveation-based Deep Video Compression without Motion Search

Chen¹,

Webb²,

Bovik³

2022

Preprint

Self Cite

View full text Add to dashboard Cite

Virtual Reality (VR) and its applications have attracted significant and increasing attention. However, the requirements of much larger file sizes, different storage formats, and immersive viewing conditions pose significant challenges to the goals of acquiring, transmitting, compressing, and displaying high-quality VR content. At the same time, the great potential of deep learning to advance progress on the video compression problem has driven a significant research effort. Because of the high bandwidth requirements of VR, there has also been significant interest in the use of space-variant, foveated compression protocols. We have integrated these techniques to create an endto-end deep learning video compression framework. A feature of our new compression model is that it dispenses with the need for expensive search-based motion prediction computations. This is accomplished by exploiting statistical regularities inherent in video motion expressed by displaced frame differences. Foveation protocols are desirable since, unlike traditional flat-panel displays, only a small portion of a video viewed in VR may be visible as a user gazes in any given direction. Moreover, even within a current field of view (FOV), the resolution of retinal neurons rapidly decreases with distance (eccentricity) from the projected point of gaze. In our learning based approach, we implement foveation by introducing a Foveation Generator Unit (FGU) that generates foveation masks which direct the allocation of bits, significantly increasing compression efficiency while making it possible to retain an impression of little to no additional visual loss given an appropriate viewing geometry. Our experiment results reveal that our new compression model, which we call the Foveated MOtionless VIdeo Codec (Foveated MOVI-Codec), is able to efficiently compress videos without computing motion, while outperforming foveated version of both H.264 and H.265 on the widely used UVG dataset and on the HEVC Standard Class B Test Sequences. The Foveated MOVI-Codec project page can be found at https://github.com/Meixu-Chen/Foveated-MOVI-Codec.

show abstract

Section: A Frameworkmentioning

confidence: 99%

Section: Foveation Generator Unit (Fgu)mentioning

confidence: 99%

See 1 more Smart Citation

Foveation-based Deep Video Compression without Motion Search

Chen¹,

Webb²,

Bovik³

2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Recent works have introduced more sophisticated components, e.g. Golinski et al [15] ture space in NVC [20], and Chen et al [9] replaced optical flow and warping by displaced frame differences. B-frame coding: Wu et al [43] introduced one of the pioneering neural video codecs via frame interpolation that was facilitated by context information.…”

Section: Related Workmentioning

confidence: 99%

Extending Neural P-frame Codecs for B-frame Coding

Pourreza¹,

Cohen²

2021

Preprint

View full text Add to dashboard Cite

While most neural video codecs address P-frame coding (predicting each frame from past ones), in this paper we address B-frame compression (predicting frames using both past and future reference frames). Our B-frame solution is based on the existing P-frame methods. As a result, B-frame coding capability can easily be added to an existing neural codec. The basic idea of our B-frame coding method is to interpolate the two reference frames to generate a single reference frame and then use it together with an existing Pframe codec to encode the input B-frame. Our studies show that the interpolated frame is a much better reference for the P-frame codec compared to using the previous frame as is usually done. Our results show that using the proposed method with an existing P-frame codec can lead to 28.5% saving in bit-rate on the UVG dataset compared to the Pframe codec while generating the same video quality.

show abstract

Tensor Spectral Pyramid for Color Video Sequences Representation, Based on 3D FO-AHKLT

Kountchev

Kountcheva²

2022

New Approaches for Multidimensional Signal Processing

View full text Add to dashboard Cite

Learning to compress videos without computing motion

Cited by 6 publications

References 34 publications

Foveation-based Deep Video Compression without Motion Search

Foveation-based Deep Video Compression without Motion Search

Extending Neural P-frame Codecs for B-frame Coding

Tensor Spectral Pyramid for Color Video Sequences Representation, Based on 3D FO-AHKLT

Contact Info

Product

Resources

About