No-reference image and video quality assessment: a classification and review of recent approaches

Abstract

The field of perceptual quality assessment has gone through a wide range of developments and it is still growing. In particular, the area of no-reference (NR) image and video quality assessment has progressed rapidly during the last decade. In this article, we present a classification and review of latest published research work in the area of NR image and video quality assessment. The NR methods of visual quality assessment considered for review are structured into categories and subcategories based on the types of methodologies used for the underlying processing employed for quality estimation. Overall, the classification has been done into three categories, namely, pixel-based methods, bitstream-based methods, and hybrid methods of the aforementioned two categories. We believe that the review presented in this article will be helpful for practitioners as well as for researchers to keep abreast of the recent developments in the area of NR image and video quality assessment. This article can be used for various purposes such as gaining a structured overview of the field and to carry out performance comparisons for the state-of-the-art methods. Keywords:No-reference; Image quality assessment; Video quality assessment; Perceptual quality 1 Review IntroductionThere has been a tremendous progress recently in the usage of digital images and videos for an increasing number of applications. Multimedia services that have gained wide interest include digital television broadcasts, video streaming applications, and real-time audio and video services over the Internet. The global mobile data traffic grew by 81% in 2013, and during 2014, the number of mobileconnected devices will exceed the number of people on earth, according to predictions made by Cisco. The video portion of the mobile data traffic was 53% in 2013 and is expected to exceed 67% by 2018 [1]. With this huge increase in the exposure of image and video to the human eye, the interest in delivering quality of experience (QoE) may increase naturally. The quality of visual media can get degraded during capturing, compression, transmission, reproduction, and displaying due to the distortions that might occur at any of these stages. The legitimate judges of visual quality are humans as end users, the opinions of whom can be obtained by subjective experiments. Subjective experiments involve a panel of participants which are usually non-experts, also referred to as test subjects, to assess the perceptual quality of given test material such as a sequence of images or videos. Subjective experiments are typically conducted in a controlled laboratory environment. Careful planning and several factors including assessment method, selection of test material, viewing conditions, grading scale, and timing of presentation have to be considered prior to a subjective experiment. For example, Recommendation (ITU-R) BT.500 [2] provides detailed guidelines for conducting subjective experiments for the assessment of quality of television pictures. The outcomes of a subjectiv...

On subjective quality assessment of adaptive video streaming via crowdsourcing and laboratory based experiments

“…The Darkhour video (51,28) is from a thriller movie and features scenes with rapid scene changes and cloudy atmosphere. The Streetdance video (46,34) is from a drama movie and consist mostly of scenes with smooth motion with static background and some scenes with groups of dancing people in bright ambient. The Football video (56,29) is from a TV broadcast of a football match and has moderate motion and wide angle camera sequences with panning.…”

“…Objective or computer software assisted methods [34] have been largely seen as an alternative approach, to get around the complications involved in the laboratory based subjective experiments. However, even the objective methods with state-of-the-art performance are not considered as an optimal replacement of subjective assessment.…”

Cognitive no-reference video quality assessment for mobile streaming services

“…In a more recent classification, Shahid et al [7] added to this classification a third category in which approaches combining both pixel and bitstream assessments are included, i.e. Hybrid NR-P-B metrics.…”

“…Due to its subjective essence, the legitimate judges of visual quality are the humans, whose opinion can be obtained through subjective analyses [7] [8]. In practice, presented stimuli (for example impaired video sequences) are rated by subjects under controlled conditions [9].…”

Challenges of future multimedia QoE monitoring for internet service providers

“…RR models receive an auxiliary channel of information, whereas NR models only inspect received signals. They therefore require less data, which typically comes at the cost of lower accuracy [38].…”