Image Retrieval Using Convolutional Autoencoder, InfoGAN, and Vision Transformer Unsupervised Models

Abstract

Query by Image Content (QBIC), subsequently known as Content-Based Image Retrieval (CBIR) systems, may offer a more advantageous solution in a variety of applications, including medical, meteorological, search by image, and others. Such systems primarily use similarity matching algorithms to compare image content to get their relevance from databases. They are essentially measuring the spatial distance between extracted visual features from a query image and their correspondence in the dataset. One of the most challenging query retrieval problems is Facial Sketched-Real Image Retrieval (FSRIR), which is content similarity matching based. These facial retrieving systems are employed in a variety of contexts, including criminal justice. The difficulties of retrieving such sorts come from the composition of the human face and its distinctive parts. In addition, the comparison between these images is made from two different domains. Besides, to our knowledge, there is a rare existence of large-scale facial datasets that can be used to evolve the performance of the retrieving system. The success of the retrieval process is governed by the method used to calculate similarity and the efficient representation of compared images. However, by effectively representing visual features, the main challenge-posing component of such approaches might be resolved. Hence, this paper has several contributions that fill the research gap in content-based similarity matching and retrieving as follows: 1) The first contribution is extending the Chinese University Face Sketch (CUFS) dataset by including augmented images, introducing to the community a novel dataset named Extended Sketched-Real Image Retrieval (ESRIR). The CUFS dataset has been extended from 100 images to include 53,000 facial sketches and 53,000 real facial images. 2) The paper's second contribution is proposing three new algorithms for sketched-real image retrieving based on convolutional autoencoder, Infogan, and Vision Transformer unsupervised models for large datasets. 3) Furthermore, to meet the subjective demands of the users because of the prevalence of multiple query formats. The third contribution of this paper is to train and assess the proposed algorithms across two additional facial datasets of various image sorts. 4) Recently, the majority of people have preferred searching for brand logo images, but it may be tricky to separate certain brand logo features from their alternatives and even from other features in an image. Thus, the fourth contribution is to compare logo image retrieval performance based on visual features derived from each of the three suggested retrieving systems. 5) The paper also proposes cloud-based energysaving and computational complexity approaches in large-scale datasets. Due to the ubiquity of touchscreen devices, users often make drawings based on their fantasies for certain object image searches. Thus, the proposed algorithms were tested and assessed on a tough dataset of doodle-scratched human artworks. They are also studied for a...

Delver Hamed : Content Based Image Retrieval Using BEIT Transformer and Detic

“…Database images which are out of interval are ignored which causes the model's speed up. Recent CBIR systems use convolutional layers as deep learning architecture for feature extraction [2], [5], [8], [13]. Some of The most Recent CBIR systems use Transformers for feature extraction that has led to satisfactory results [3], [4], [6], [7], [8].…”

“…Recent CBIR systems use convolutional layers as deep learning architecture for feature extraction [2], [5], [8], [13]. Some of The most Recent CBIR systems use Transformers for feature extraction that has led to satisfactory results [3], [4], [6], [7], [8]. Transformers have made a big evolution in AI even in image processing.…”

“…Transformers have made a big evolution in AI even in image processing. [4], [6], [7] and [8] used Vision Transformer as a feature extractor.…”

“…[6] Used Vison Transformer with metric learning objective. [8] Used Vision Transformer for Sketched-Real Image Retrieval (SRIR) beside Info-GAN on ESRIR dataset. I used BEIT Transformer that gained better top1-accuracy than Vision Transformer in ImageNet dataset for classification task [15].…”

Image Insertion Using Depth- and Topology- Constrained Minimum Spanning Tree for Compressed Image Sets

“…smart phones and tablets. With their wide uses, a great number of digital photos have been taken by people anytime and anywhere, which are usually delivered to and then stored in cloud servers [1], [2], [3], [4], [5]. For any image in the enormous amount of photos, there may be some other images with similar contents, which thus can be called similar images.…”