Subjective evaluation of JPEG XR image compression

Abstract

In this paper a procedure for subjective evaluation of the new JPEG XR codec for compression of still pictures is described in details. The new algorithm has been compared to the existing JPEG and JPEG 2000 standards when considering compression of high resolution 24 bpp pictures, by mean of a campaign of subjective quality assessment tests which followed the guidelines defined by the AIC JPEG ah-hoc group. Sixteen subjects took part in experiments at EPFL and each subject participated in four test sessions, scoring a total of 208 test stimuli. A detailed procedure for statistical analysis of subjective data is also proposed and performed. The obtained results show high consistency and allow an accurate comparison of codec performance.

Calculation of average coding efficiency based on subjective quality scores

“…During the development phase of their compression standards, Joint Photographic Experts Group (JPEG), Moving Picture Experts Group (MPEG), and Video Coding Experts Group (VCEG) have relied during past years on both objective and subjective evaluations to select and evaluate potential coding technologies, as well as for verification purposes. For example, subjective evaluations were conducted during the development of JPEG XR [3], MPEG-4 [4], H.264/MPEG-4 AVC [5,6,7], Scalable Video Coding (SVC) extension of H.264/MPEG-4 AVC [8,7], and H.265/HEVC [9,7]. Independent researchers have also conducted subjective evaluations, both during and after the development phase of compression standards, as a validation process or to evaluate the codecs in different scenarios.…”

Overview of the JPEG XS core coding system subjective evaluations

“…The JPEG XR image compression standard 16 was evaluated using the Double Stimulus Continuous Quality Scale (DSCQS) method, aptly modified for the evaluation of still images. Specifically, a side-by-side comparison with implicit reference was implemented.…”

Performance analysis of VP8 image and video compression based on subjective evaluations

“…The first implementation is the JPEG XR reference software § , version 1.20. The second one is that provided by Microsoft for the study in 7 and includes a pre-processing tool to define content adaptive quantization tables 15 (we will refer to this configuration as xr(ms) in the rest of the paper). For the JPEG XR reference software implementation, the quantization steps for the chrominance channels were derived from the luminance channel quantization steps.…”

“…Existing studies on the performance evaluation of JPEG XR show that the new standard achieves significantly better compression efficiency than JPEG, with overall performance slighlty below or comparable to JPEG 2000 . 7 Two implementations of the JPEG XR codec have been considered. The first implementation is the JPEG XR reference software § , version 1.20.…”

“…Examples of recent efforts to define new international compression standards, with higher coding efficiency than the state of the art solutions, are the JPEG XR image compression standard, approved and published as ITU-T recommendation and ISO/IEC standard in 2010, 2 bandwidth provisioning in multimedia oriented networks and systems. They are conducted either during the development and standardization of new codecs to assess the improved coding efficiency with respect to other existing solutions 7,8 or after the adoption to compare different codec implementations with each other.…”