Reorganized DCT-based image representation for reduced reference stereoscopic image quality assessment

Ma, Lin; Wang, Xu; Liu, Qiong; Ngan, King Ngi

doi:10.1016/j.neucom.2015.06.116

Cited by 50 publications

(25 citation statements)

References 57 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In our previous work [39][40][41] , the difference image calculated from left and right views has been demonstrated to retain stereoscopic perception information, which can be used to represent the quality of stereoscopic image. Additionally, Ma et al [42] concluded that the difference image is more valuable than the left and right views in SIQA task. Similarly, we evaluate stereoscopic video quality by conducting our 3D CNN on the difference video rather than directly on the left and right views in this work.…”

Section: Difference Videomentioning

confidence: 99%

Stereoscopic video quality assessment based on 3D convolutional neural networks

Yang

Zhu

et al. 2018

Neurocomputing

View full text Add to dashboard Cite

Keywords:3D convolutional neural networks Stereoscopic video quality assessment Quality score fusion a b s t r a c tThe research of stereoscopic video quality assessment (SVQA) plays an important role for promoting the development of stereoscopic video system. Existing SVQA metrics rely on hand-crafted features, which is inaccurate and time-consuming because of the diversity and complexity of stereoscopic video distortion. This paper introduces a 3D convolutional neural networks (CNN) based SVQA framework that can model not only local spatio-temporal information but also global temporal information with cubic difference video patches as input. First, instead of using hand-crafted features, we design a 3D CNN architecture to automatically and effectively capture local spatio-temporal features. Then we employ a quality score fusion strategy considering global temporal clues to obtain final video-level predicted score. Extensive experiments conducted on two public stereoscopic video quality datasets show that the proposed method correlates highly with human perception and outperforms state-of-the-art methods by a large margin. We also show that our 3D CNN features have more desirable property for SVQA than hand-crafted features in previous methods, and our 3D CNN features together with support vector regression (SVR) can further boost the performance. In addition, with no complex preprocessing and GPU acceleration, our proposed method is demonstrated computationally efficient and easy to use. (W. Lu).Unfortunately, considering that reference video is unavailable in most practical applications, only the NR methods have potential to satisfy the actual requirement. As a result, our work focuses on more appealing and challenging NR methods, and tries to propose a new general-purpose NR framework for SVQA.Considering the development of NR metrics [1,2] , we can find that most of them have similar frameworks, which can be generally divided into two steps: (1) extracts features that can reflect visual quality based on relatively perceptual models; (2) maps the obtained feature vectors to subjective quality scores by learning regression models. But in real-world scenarios, because of the diversity and complexity of video distortions, it is very difficult to identify what features are sensitive and robust to all sorts of distortions. Therefore, simply utilizing a group of artificially designed features to represent video quality results in inaccurate assessments and high computational costs.Recently, it is a well-known fact that deep learning models, especially convolutional neural networks (CNN), have achieved great success in many challenge computer vision tasks, such as image classification [3,4] , object detection [5,6] , video classification [7,8] and video action recognition [9,10] . CNN is a biologically inspired architecture consisting of a stack of convolutional layers

show abstract

Section: Difference Videomentioning

confidence: 99%

Stereoscopic video quality assessment based on 3D convolutional neural networks

Yang

Zhu

et al. 2018

Neurocomputing

View full text Add to dashboard Cite

show abstract

“…As mentioned above, the view difference image is very important for representing the correlation between views because both the image texture information and depth information of the stereoscopic pairs [25] are included in the view difference image. Therefore, we establish a modified CNN, the pooling layers and fully connected layers of which are removed according to the SR task to construct the high-precision view difference image, as shown in Figure 4.…”

Section: Estimation Of View Difference Image With Modified Cnnmentioning

confidence: 99%

“…Thus, the disparity map cannot be accurately identified. Similar to [25], the view difference image is directly calculated from the stereoscopic image pairs as:…”

Section: Image Patch Extraction and Representationmentioning

confidence: 99%

Stereoscopic Image Super-Resolution Method with View Incorporation and Convolutional Neural Networks

Pan

Jiang

et al. 2017

Applied Sciences

View full text Add to dashboard Cite

Super-resolution (SR) plays an important role in the processing and display of mixed-resolution (MR) stereoscopic images. Therefore, a stereoscopic image SR method based on view incorporation and convolutional neural networks (CNN) is proposed. For a given MR stereoscopic image, the left view of which is observed in full resolution, while the right view is viewed in low resolution, the SR method is implemented in two stages. In the first stage, a view difference image is defined to represent the correlation between views. It is estimated by using the full-resolution left view and the interpolated right view as input to the modified CNN. Accordingly, a high-precision view difference image is obtained. In the second stage, to incorporate the estimated right view in the first stage, a global reconstruction constraint is presented to make the estimated right view consistent with the low-resolution right view in terms of the MR stereoscopic image observation model. Experimental results demonstrated that, compared with the SR convolutional neural network (SRCNN) method and depth map based SR method, the proposed method improved the reconstructed right view quality by 0.54 dB and 1.14 dB, respectively, in the Peak Signal to Noise Ratio (PSNR), and subjective evaluation also implied that the proposed method produced better reconstructed stereoscopic images.

show abstract

“…Inspired by the cyclopean image, SIQA has been further developed. In [26], an RR-IQA method was proposed based on the statistical properties of the stereoscopic image in the reorganized discrete cosine transform domain. Shao et al trained two deep neural networks from monocular images and cyclopean images to model the process of quality prediction [20].…”

Section: Introductionmentioning

confidence: 99%

Sparse representation based stereoscopic image quality assessment accounting for perceptual cognitive process

Yang

Jiang

Wang

et al. 2018

Information Sciences

View full text Add to dashboard Cite

In this paper, we propose a sparse representation based Reduced-Reference Image Quality Assessment (RR-IQA) index for stereoscopic images from the following two perspectives: 1) Human visual system (HVS) always tries to infer the meaningful information and reduces uncertainty from the visual stimuli, and the entropy of primitive (EoP) can well describe this visual cognitive progress when perceiving natural images. 2) Ocular dominance (also known as binocularity) which represents the interaction between two eyes is quantified by the sparse representation coefficients. Inspired by previous research, the perception and understanding of an image is considered as an active inference process determined by the level of "surprise", which can be described by EoP. Therefore, the primitives learnt from natural images can be utilized to evaluate the visual information by computing entropy. Meanwhile, considering the binocularity in stereo image quality assessment, a feasible way is proposed to characterize this binocular process according to the sparse representation coefficients of each view. Experimental results on LIVE 3D image databases and MCL database further demonstrate that the proposed algorithm achieves high consistency with subjective evaluation.

show abstract

Reorganized DCT-based image representation for reduced reference stereoscopic image quality assessment

Cited by 50 publications

References 57 publications

Stereoscopic video quality assessment based on 3D convolutional neural networks

Stereoscopic video quality assessment based on 3D convolutional neural networks

Stereoscopic Image Super-Resolution Method with View Incorporation and Convolutional Neural Networks

Sparse representation based stereoscopic image quality assessment accounting for perceptual cognitive process

Contact Info

Product

Resources

About