Blind image quality assessment based on the multiscale and dual‐domains features fusion

Abstract

Image quality assessment is to simulate subjective human visual perception and realize image quality inference automatically. Although deep neural networks have achieved great success, the majority of them do not fully consider perception characteristics. Therefore, according to the human visual scale characteristics, we proposed an image quality assessment algorithm based on multiscale and dual domains fusion. Firstly, the original image and its phase congruency respectively input into two branches, feature pyramid and channel attention mechanism are adopted to extract multiscale features. After that, bilinear pool is used to aggregate the spatial and frequency domain characteristics of the corresponding scales, and allows arbitrary scale input to ensure that the features are extracted from the inherent quality images. Finally, the single quality score is obtained through learned weights of each scale. Comparative experiments between our approach and state-of-the-art are conducted on five public databases, the results demonstrate that the proposed algorithm is not only robust to different types and across database, but also sensitive to scale. K E Y W O R D Simage quality assessment, multiscale features, original scale input, phase congruency INTRODUCTIONPeople have higher and higher requirements for image quality, so it is an important research topic to automatically and accurately assess image quality before display 1 to ensure human visual perception. Image quality assessment (IQA) can be used to optimize the process of image acquisition, transmission, and storage, thereby minimizing possible image quality degradation. It can also be used to guide image restoration, image deblurring, image super-resolution reconstruction, and other tasks to improve image quality. 2According to the needs of reference information, IQA methods can be divided into full reference (FR), reduced reference (RR), and no reference (NR). FR-IQA calculates the difference between the reference image and the distorted image to obtain the quality score of the distorted image. The classic algorithms are structural similarity (SSIM) 3 and feature similarity (FSIM). 4 The Human Visual System (HVS) is suitable for the extraction of structural information, and image distortion can also change the perception of structural information. Therefore, SSIM calculates the illuminance, contrast and structural similarity between the reference image and the test image to obtain quality index. However, when pooling into a single image quality score, the weights of different local regions are not considered. FSIM proposed that HVS is mainly based on low-level features for image understanding. Therefore, the low-level feature similarity is proposed to obtain the quality index. Specifically, the phase consistency (PC) and the gradient magnitude (GM) are complementary, and the PC is also used as a weight to obtain the similarity score.

Artificial Intelligence Auxiliary Algorithm for Wushu Routine Competition Decision Based on Feature Fusion

“…Initially, the collected Wush images are preprocessed to eliminate background noise and interference. Then, multifeature extraction methods [ 26 – 28 ] are applied to extract discriminant feature patterns from the preprocessed images. The extracted feature patterns are fused and provided as input to the proposed CNN model.…”

Deep Superpixel-Based Network For Blind Image Quality Assessment

“…This network is aware of both the semantic contents and the quality of images. Inspired by earlier works [8,9],we apply a multiscale features extraction model in our backbone network. In this way, the local contents and distortions are extracted more completely.…”

“…In order to design a BIQA method that is more consistent with human perception, we propose a deep superpixel-based network for BIQA (DSN-IQA). Following the previous work [8,9], we develope our CNN based network that extracts multi-scale semantic features, and fuse these features with the superpixel adjacency map obtained from superpixel segmentation model. Because when human eyes evaluate an image, they will pay attention to the semantic information and local details of the image at the same time and finally get the quality.…”

Computer Vision‐Driven Evaluation System for Assisted Decision‐Making in Sports Training

“…In recent years, with the emergence of big data and the substantial increase in computer parallel computing capabilities, deep learning [11][12][13][14] has made breakthroughs in the fields of computer vision [15][16][17] and natural language processing by relying on rich training data and powerful feature expression capabilities [18]; especially in the goals, many practical breakthroughs have been made in detection, machine translation [19], and motion recognition. Target detection is an important research topic of computer vision.…”