MobileCodec

Le, Hoang; Zhang, Liang; Said, Amir; Sautiere, Guillaume; Yang, Yang; Shrestha, Pranav; Yin, Fei; Pourreza, Reza; Wiggers, Auke

doi:10.1145/3524273.3532906

Cited by 16 publications

(4 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This result demonstrates that after adopting the calibration strategy, our model can successfully decode all video frames in cross-platform scenarios. Furthermore, we compare our model's performance with MobileCodec [49], which is the first-ever inter-frame neural video decoder running on a commercial mobile phone but does not account for the cross-platform issue. The green star in the figure represents the result of MobileCodec.…”

Section: Real-time Cross-platform Resultsmentioning

confidence: 99%

“…AlphaVC introduces several techniques, e.g., conditional I-frame and pixel-to-feature motion prediction, to improve the rate-distortion performance [37]. There is also a work of real-time video decoding, called MobileCodec, which is the first-ever inter-frame neural video decoder running on a commercial mobile phone taking no account of the cross-platform issue [49].…”

Section: Neural Video Compressionmentioning

confidence: 99%

“…Motion Estimation Downsampling. MobileCodec replaces the motion compensation by using a fully convolutional network to avoid warping [49]. Unlike this, in order to reduce the computational complexity of motion estimation, we downsample the input image of the motion estimation network by a factor of 2 from the original size, which reduces the amount of calculation to 1/4 compared with the original setting.…”

Section: Computing Efficiency Improvementmentioning

confidence: 99%

See 2 more Smart Citations

Towards Real-Time Neural Video Codec for Cross-Platform Application Using Calibration Information

Tian,

Guan,

Xiang

et al. 2023

Proceedings of the 31st ACM International Conference on Multimedia

View full text Add to dashboard Cite

The state-of-the-art neural video codecs have outperformed the most sophisticated traditional codecs in terms of rate-distortion (RD) performance in certain cases. However, utilizing them for practical applications is still challenging for two major reasons. 1) Cross-platform computational errors resulting from floating point operations can lead to inaccurate decoding of the bitstream. 2) The high computational complexity of the encoding and decoding process poses a challenge in achieving real-time performance. In this paper, we propose a real-time cross-platform neural video codec, which is capable of efficiently decoding (≈25FPS) of 720P video bitstream from other encoding platforms on a consumer-grade GPU (e.g., NVIDIA RTX 2080). First, to solve the problem of inconsistency of codec caused by the uncertainty of floating point calculations across platforms, we design a calibration transmitting system to guarantee the consistent quantization of entropy parameters between the encoding and decoding stages. The parameters that may have transboundary quantization between encoding and decoding are identified in the encoding stage, and their coordinates will be delivered by auxiliary transmitted bitstream. By doing so, these inconsistent parameters can be processed properly in the decoding stage. Furthermore, to reduce the bitrate of the auxiliary bitstream, we rectify the distribution of entropy parameters using a piecewise Gaussian constraint. Second, to match the computational limitations on the decoding side for real-time video codec, we design a lightweight model. A series of efficiency techniques, such as model pruning, motion downsampling, and arithmetic coding skipping, enable our model to achieve 25 FPS decoding speed on NVIDIA RTX 2080 GPU. Experimental results demonstrate that our model can achieve real-time decoding of 720P videos while encoding on another platform. Furthermore, the real-time model brings up to a maximum of 24.2% BD-rate improvement from the perspective of PSNR with the anchor H.265 (medium).

show abstract

Section: Real-time Cross-platform Resultsmentioning

confidence: 99%

Section: Neural Video Compressionmentioning

confidence: 99%

Section: Computing Efficiency Improvementmentioning

confidence: 99%

See 1 more Smart Citation

Towards Real-Time Neural Video Codec for Cross-Platform Application Using Calibration Information

Tian,

Guan,

Xiang

et al. 2023

Proceedings of the 31st ACM International Conference on Multimedia

View full text Add to dashboard Cite

show abstract

“…However, they are currently impractical on small devices because of the computational complexity and energy consumption. It has been reported that a neural decoder needs more than 1000 times the number of MACs/pixel compared to standard codecs [11] [12]. Even with network quantization and distillation like in [13], the necessary number of MACs can be around 500 times more compared to standard codecs.…”

Section: Introductionmentioning

confidence: 99%

Entropy Coding Improvement for Low-complexity Compressive Auto-encoders

Galpin¹,

Balcilar²,

Lefèbvre³

et al. 2023

Preprint

View full text Add to dashboard Cite

End-to-end image and video compression using auto-encoders (AE) offers new appealing perspectives in terms of rate-distortion gains and applications. While most complex models are on par with the latest compression standard like VVC/H.266 on objective metrics, practical implementation and complexity remain strong issues for real-world applications. In this paper, we propose a practical implementation suitable for realistic applications, leading to a low-complexity model. We demonstrate that some gains can be achieved on top of a state-of-the-art low-complexity AE, even when using simpler implementation. Improvements include off-training entropy coding improvement and encoder side Rate Distortion Optimized Quantization. Results show a 19% improvement in BDrate on basic implementation of fully-factorized model, and 15.3% improvement compared to the original implementation. The proposed implementation also allows a direct integration of such approaches on a variety of platforms.

show abstract