Overview of the Screen Content Support in VVC: Applications, Coding Tools, and Performance

Abstract

In an increasingly connected world, consumer video experiences have diversified away from traditional broadcast video into new applications with increased use of non-cameracaptured content such as computer screen desktop recordings or animations created by computer rendering, collectively referred to as screen content. There has also been increased use of graphics and character content that is rendered and mixed or overlaid together with camera-generated content. The emerging Versatile Video Coding (VVC) standard, in its first version, addresses this market change by the specification of low-level coding tools suitable for screen content. This is in contrast to its predecessor, the High Efficiency Video Coding (HEVC) standard, where highly efficient screen content support is only available in extension profiles of its version 4. This paper describes the screen content support and the five main low-level screen content coding tools in VVC: transform skip residual coding (TSRC), block-based differential pulse-code modulation (BDPCM), intra block copy (IBC), adaptive color transform (ACT), and the palette mode. The specification of these coding tools in the first version of VVC enables the VVC reference software implementation (VTM) to achieve average bit-rate savings of about 41% to 61% relative to the HEVC test model (HM) reference software implementation using the Main 10 profile for 4:2:0 screen content test sequences. Compared to the HM using the Screen-Extended Main 10 profile and the same 4:2:0 test sequences, the VTM provides about 19% to 25% bit-rate savings. The same comparison with 4:4:4 test sequences revealed bit-rate savings of about 13% to 27% for Y CBCR and of about 6% to 14% for R G B screen content. Relative to the HM without the HEVC version 4 screen content coding extensions, the bit-rate savings for 4:4:4 test sequences are about 33% to 64% for Y CBCR and 43% to 66% for R G B screen content.

Transform Skip Inspired End-to-End Compression for Screen Content Image

“…Generally speaking, screen content is an umbrella-term of the computer-generated or rendered data characterized with noise-free, sharp-edges and high contrast [1]. In the past decades, a series of efforts have been dedicated to improve the compression performance of the screen content.…”

“…The screen content coding tools could seamlessly collaborate with the scene characteristic and the coding characteristic of screen content. It was reported that the screen content coding tools could bring additional 33.22% BD-Rate savings [1], laying the foundation of versatile usage of the VVC.…”

Enhanced Intra String Copy for Screen Content Coding in AVS3

“…Based on the observation and minimum entropy principle, value range C is divided into multiple subranges for optimal coding and binarization. For the string lengths of Evs and Ups, there are eleven subranges, which are the intersection of C and [0,1), [1,2), [2,4), [4,8), [8,16), [16,32), [32,64), [64,128), [128,256), [256,512), and [512,1024), respectively, that is,…”

“…With more and more popular applications using computer screen as interfaces for daily remote human-machine and human-human interactions, such as online education, teleconferencing, telecommuting, and remote entertainment, almost all recent video coding standards have included screen content coding (SCC) tools [1]. The video coding standards with SCC tools include high efficiency video coding (HEVC), versatile video coding (VVC) [2], the secondgeneration audio video standard (AVS2), the third-generation AVS (AVS3) [3], [4], AV1, and essential video coding (EVC).…”

“…In various standards, the adopted SCC tools are significantly different [1]. Five main SCC tools have been adopted in VVC [2]: transform skip residual coding, block-based differential pulse-code modulation (BDPCM), intra-block copy (IBC), adaptive color transform (ACT), and palette. In AVS3, five main SCC tools have been adopted in the draft [5]: IBC [6], frequency-based intra-mode coding (FIMC) [7], implicit selection of transform skip (ISTS) [8], adaptive control of deblocking type (ACDT) [9], and intra string copy (ISC) [10]- [12].…”