An Optical Flow-Based Full Reference Video Quality Assessment Algorithm

Abstract

We present a simple yet effective optical flow-based full-reference video quality assessment (FR-VQA) algorithm for assessing the perceptual quality of natural videos. Our algorithm is based on the premise that local optical flow statistics are affected by distortions and the deviation from pristine flow statistics is proportional to the amount of distortion. We characterize the local flow statistics using the mean, the standard deviation, the coefficient of variation (CV), and the minimum eigenvalue ( λ min ) of the local flow patches. Temporal distortion is estimated as the change in the CV of the distorted flow with respect to the reference flow, and the correlation between λ min of the reference and of the distorted patches. We rely on the robust multi-scale structural similarity index for spatial quality estimation. The computed temporal and spatial distortions, thus, are then pooled using a perceptually motivated heuristic to generate a spatio-temporal quality score. The proposed method is shown to be competitive with the state-of-the-art when evaluated on the LIVE SD database, the EPFL Polimi SD database, and the LIVE Mobile HD database. The distortions considered in these databases include those due to compression, packet-loss, wireless channel errors, and rate-adaptation. Our algorithm is flexible enough to allow for any robust FR spatial distortion metric for spatial distortion estimation. In addition, the proposed method is not only parameter-free but also independent of the choice of the optical flow algorithm. Finally, we show that the replacement of the optical flow vectors in our proposed method with the much coarser block motion vectors also results in an acceptable FR-VQA algorithm. Our algorithm is called the flow similarity index.

A no-reference optical flow-based quality evaluator for stereoscopic videos in curvelet domain

“…The success of VQA algorithms should depend on their abilities to model motion perception in the HVS. Manasa et al [26] estimated the temporal distortion by the use of the mean, the standard deviation, the coefficient of variation, and the minimum eigenvalue of local optical flow statistics. In addition, experiments demonstrated that local flow statistics are effective features for assessing temporal quality.…”

Audiovisual quality of live music streaming over mobile networks using MPEG-DASH

“…Regarding the objective quality metrics for audio, these include PEAQ (Perceived Evaluation of Audio Quality) [14], POLQA Music (Perceptual Objective Listening Quality Assessment) [15,16], and ViSQOL Audio (Virtual Speech Quality Objective Listener) [17]. For video, a whole range of quality metrics exist, such as PEVQ (Perceptual Evaluation of Video Quality) [18], VQM (Video Quality Metric) [19], ST-MAD (Spatiotemporal Most-Apparent Distortion model) [20], MOVIE (Motion-based Video Integrity Evaluation) [21], ST-RRED (Spatiotemporal Reduced Reference Entropic Differences) [22], and FLOSIM [23], among others. It is also common to adapt Image Quality Metrics, such as PSNR (Peak Signal-to-Noise Ratio) and MS-SSIM [24] using the average of frame-wise measurements.…”

A machine-learning-driven solution to the problem of perceptual video quality metrics

“…Various other FR metrics have been proposed in the literature [12]. Aabed et al [13] proposed a perceptual quality metric that utilises low complexity power spectral features in the frequency domain, while Manasa and Channappayya [14] proposed the use of optical flow statistics, such as the minimum eigenvalue of the local flow patches, the mean, the standard deviation, and the coefficient of variation in order to estimate temporal quality and the use of SSIM for spatial quality estimation, combining both for computing the final quality score. Seshadrinathan and Bovik [15] proposed the MOVIE index that examines temporal, spatial, and spatiotemporal characteristics of distortion in order to estimate video quality.…”