Overview of Screen Content Video Coding: Technologies, Standards, and Beyond

Peng, Wen-Hsiao; Walls, Frederick; Cohen, Robert; Xu, Jizheng; Östermann, Jörn; MacInnis, Alexander; Lin, Tao

doi:10.1109/jetcas.2016.2608971

Cited by 60 publications

(17 citation statements)

References 71 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Non-photographic images are images that are different than the natural continuous-tone images, e.g., they are computer-generated, composed from others (including natural ones), or are screenshots. They may be described as screen-content images and it is worth mentioning that, although photographic images are the most common in real-life PACSs, there currently is a growing interest in compression of screen content images [ 36 ].…”

Section: Methodsmentioning

confidence: 99%

Hybrid Adaptive Lossless Image Compression Based on Discrete Wavelet Transform

Starosolski¹

2020

Entropy

View full text Add to dashboard Cite

A new hybrid transform for lossless image compression exploiting a discrete wavelet transform (DWT) and prediction is the main new contribution of this paper. Simple prediction is generally considered ineffective in conjunction with DWT but we applied it to subbands of DWT modified using reversible denoising and lifting steps (RDLSs) with step skipping. The new transform was constructed in an image-adaptive way using heuristics and entropy estimation. For a large and diverse test set consisting of 499 photographic and 247 non-photographic (screen content) images, we found that RDLS with step skipping allowed effectively combining DWT with prediction. Using prediction, we nearly doubled the JPEG 2000 compression ratio improvements that could be obtained using RDLS with step skipping. Because for some images it might be better to apply prediction instead of DWT, we proposed compression schemes with various tradeoffs, which are practical contributions of this study. Compared with unmodified JPEG 2000, one scheme improved the compression ratios of photographic and non-photographic images, on average, by 1.2% and 30.9%, respectively, at the cost of increasing the compression time by 2% and introducing only minimal modifications to JPEG 2000. Greater ratio improvements, exceeding 2% and 32%, respectively, are attainable at a greater cost.

show abstract

Section: Methodsmentioning

confidence: 99%

Hybrid Adaptive Lossless Image Compression Based on Discrete Wavelet Transform

Starosolski¹

2020

Entropy

View full text Add to dashboard Cite

show abstract

“…A number of solutions for encoding mixed content images and video (sometimes also referred to as compound images and compound video) such as computer screens have been proposed over time [14]. In the following, we review the relevant literature highlighting the relative limitations that prompted the development of this work.…”

Section: Related Workmentioning

confidence: 99%

Very Low Bitrate Semantic Compression of Airplane Cockpit Screen Content

Mitrica

Moulines²,

Ruellan³

et al. 2019

IEEE Trans. Multimedia

View full text Add to dashboard Cite

This work addresses the problem of encoding the video generated by the screen of an airplane cockpit. As other computer screens, cockpit screens consists in computergenerated graphics often atop natural background. Existing screen content coding schemes fail notably in preserving the readability of textual information at the low bitrates required in avionic applications. We propose a screen coding scheme where textual information is encoded according to the relative semantics rather than in the pixel domain. The encoder localizes textual information, the semantics of each character are extracted with a convolutional neural network and are predictively encoded. Text is then removed via inpainting, the residual background video is compressed with a standard codec and transmitted to the receiver together with the text semantics. At the decoder side, text is synthesized using the decoded semantics and superimposed over the decoded residual video recovering the original frame. Our proposed scheme offers two key advantages over a semanticsunaware scheme that encodes text in the pixel domain. First, the text readability at the decoder is not compromised by compression artifacts, whereas the relative bitrate is negligible. Second, removal of high-frequency transform coefficients associated to the inpainted text drastically reduces the bitrate of the residual video. Experiments with real cockpit video sequences show BDrate gains up to 82% and 69 % over a reference H.265/HEVC encoder and its SCC extension. Moreover, our scheme achieves quasi-errorless character recognition already at very low bitrates, whereas even HEVC-SCC needs at least 3 or 4 times more bitrate to achieve a comparable error rate. Index Terms-HEVC, screen content coding, cockpit content coding, low bitrate, character recognition, semantic video coding, convolutional neural networks, compound video, compound imagesThis is the author's version of an article that has been published in this journal. Changes were made to this version by the publisher prior to publication.The final version of record is available at http://dx.

show abstract

“…After finalization of the HEVC base specification, JCT-VC continued to work on extensions. The screen content coding (SCC) extensions [3] improve compression capability for video containing a significant portion of rendered (moving or static) graphics, text, or animation rather than camera captured video scenes. In the past few years, screen content video is more and more popular in applications, such as remote desktop, video conferencing, screen sharing, and cloud computing.…”

Section: Introductionmentioning

confidence: 99%

Distortion Propagation Based Quantization Parameter Cascading Method for Screen Content Video Coding

Tang

et al. 2019

IEEE Access

View full text Add to dashboard Cite

Screen content video (SCV) is generated by computers, including animation, texts and graphics. SCV contains continuous static frames and many scene changes, making SCV different from conventional camera captured video (CCV) in terms of temporal characteristic. Therefore, conventional quantization parameter (QP) cascading method may not be efficient for SCV. In this paper, a distortion propagation based QP cascading method is proposed for SCV. The special temporal characteristic of SCV is considered and the distortion propagation of every coding tree unit (CTU) is measured. Based on the CTU level distortion propagation, the improved QP cascading method is designed. Experimental results show that compared with other methods, the proposed could achieve better rate distortion (RD) performance and less encoding time.INDEX TERMS Rate control, video coding, mobile, screen content, HEVC.

show abstract

Overview of Screen Content Video Coding: Technologies, Standards, and Beyond

Cited by 60 publications

References 71 publications

Hybrid Adaptive Lossless Image Compression Based on Discrete Wavelet Transform

Hybrid Adaptive Lossless Image Compression Based on Discrete Wavelet Transform

Very Low Bitrate Semantic Compression of Airplane Cockpit Screen Content

Distortion Propagation Based Quantization Parameter Cascading Method for Screen Content Video Coding

Contact Info

Product

Resources

About