An Image Retrieval Framework Design Analysis Using Saliency Structure and Color Difference Histogram

Chugh, Himani; Gupta, Sheifali; Garg, Meenu; Gupta, Deepali; Mohamed, Heba G.; Noya, Irene Delgado; Singh, Aman; Goyal, Nitin

doi:10.3390/su141610357

Cited by 10 publications

(6 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In this way, the authors set the detection con dence to a lower value thereby avoiding missing detections. The authors in [8] propose a tracking scheme based on ne-tuned YOLOv3 and DeepSort to monitor social distancing during COVID-19 [45]. The YOLO v3 detector separates people from the background and uses DeepSort for tracking the identi ed people using bounding boxes and IDs assigned.…”

Section: Literature Surveymentioning

confidence: 99%

SDSCF Tracker: An improved deep learning-based approach to execute target tracking

Gautam¹,

Singh

2023

Preprint

View full text Add to dashboard Cite

SDSCF tracker is a novel tracking via detection-based framework developed to address the problem of background clutter and reduced ID switches caused during tracking process. The entire mechanism is divided in three stages namely: detection stage followed by segmentation and tracking. Initially, the input sequence is fed to the improved YOLO-Lite detector to extract the primary bounding boxes. After the detection has been performed, all the objects inside the frames undergo segmentation to eliminate the redundancy and background clutter. By integrating instance segmentation post-detection and prior to tracking, the computational parameters reduce drastically for each frame. Next, the correlation filter-based tracker along with DeepSort performs object tracking. DeepSort performs very well on the frames containing detections even through lengthier stages of occlusion however, fails in case of similar looking objects resulting in ID switches. Correlation filter-based tracker counters the problem and keeps the track of each target even if it remains undetected in detection stage. New IoU’s for bounding boxes tracked by correlation tracker and bounding boxes tracked by DeepSort are computed and based on new IoU obtained, decision is made weather the target ID from DeepSort or correlation tracker is to be considered thereby preventing ID switching. Experimental evaluation carried out on MOT15, MOT16 and MOT17 shows that the proposed framework reduces the amount of ID switches by almost 53 % achieving highest MOTA and MOTP of 69.6% and 84.7% respectively.

show abstract

Section: Literature Surveymentioning

confidence: 99%

SDSCF Tracker: An improved deep learning-based approach to execute target tracking

Gautam¹,

Singh

2023

Preprint

View full text Add to dashboard Cite

show abstract

“…This will reduce the capacity to express visual content and cannot handle complex similarity semantics well. Chugh et al [22] combined multiple features to improve the retrieval of plants. With the popularity of deep convolutional neural networks (CNNs), CNNs are also gradually used for hash learning to solve the above problems.…”

Section: Related Workmentioning

confidence: 99%

Transformer-Based Distillation Hash Learning for Image Retrieval

Yuan-hai

Wang

Yuan³

et al. 2022

Electronics

View full text Add to dashboard Cite

In recent years, Transformer has become a very popular architecture in deep learning and has also achieved the same state-of-the-art performance as convolutional neural networks on multiple image recognition baselines. Transformer can obtain global perceptual fields through a self-attention mechanism and can enhance the weights of unique discriminable features for image retrieval tasks to improve the retrieval quality. However, Transformer is computationally intensive and finds it difficult to satisfy real-time requirements when used for retrieval tasks. In this paper, we propose a Transformer-based image hash learning framework and compress the constructed framework to perform efficient image retrieval using knowledge distillation. By combining the self-attention mechanism of the Transformer model, the image hash code is enabled to be global and unique. At the same time, this advantage is instilled into the efficient lightweight model by knowledge distillation, thus reducing the computational complexity and having the advantage of an attention mechanism in the Transformer. The experimental results on the MIRFlickr-25K dataset and NUS-WIDE dataset show that our approach can effectively improve the accuracy and efficiency of image retrieval.

show abstract

“…This simplification leads to a gap between the semantic richness of images and the descriptive power of the generated text [12]. The challenge is further compounded by the static nature of the training process in traditional supervised learning models, which limits their ability to adapt to new or varied data inputs dynamically [13]. Notably, recent research by Lu et al [14,15] has introduced enhanced attention mechanisms and language models to better capture the subtleties of cross-modal interactions.…”

Section: Introduction and Related Workmentioning

confidence: 99%

Enhancing Image Description Generation through Deep Reinforcement Learning: Fusing Multiple Visual Features and Reward Mechanisms

Li,

Wang,

Jia

2024

CMC

View full text Add to dashboard Cite

Image description task is the intersection of computer vision and natural language processing, and it has important prospects, including helping computers understand images and obtaining information for the visually impaired. This study presents an innovative approach employing deep reinforcement learning to enhance the accuracy of natural language descriptions of images. Our method focuses on refining the reward function in deep reinforcement learning, facilitating the generation of precise descriptions by aligning visual and textual features more closely. Our approach comprises three key architectures. Firstly, it utilizes Residual Network 101 (ResNet-101) and Faster Region-based Convolutional Neural Network (Faster R-CNN) to extract average and local image features, respectively, followed by the implementation of a dual attention mechanism for intricate feature fusion. Secondly, the Transformer model is engaged to derive contextual semantic features from textual data. Finally, the generation of descriptive text is executed through a two-layer long short-term memory network (LSTM), directed by the value and reward functions. Compared with the image description method that relies on deep learning, the score of Bilingual Evaluation Understudy (BLEU-1) is 0.762, which is 1.6% higher, and the score of BLEU-4 is 0.299. Consensus-based Image Description Evaluation (CIDEr) scored 0.998, Recall-Oriented Understudy for Gisting Evaluation (ROUGE) scored 0.552, the latter improved by 0.36%. These results not only attest to the viability of our approach but also highlight its superiority in the realm of image description. Future research can explore the integration of our method with other artificial intelligence (AI) domains, such as emotional AI, to create more nuanced and context-aware systems.

show abstract

An Image Retrieval Framework Design Analysis Using Saliency Structure and Color Difference Histogram

Cited by 10 publications

References 24 publications

SDSCF Tracker: An improved deep learning-based approach to execute target tracking

SDSCF Tracker: An improved deep learning-based approach to execute target tracking

Transformer-Based Distillation Hash Learning for Image Retrieval

Enhancing Image Description Generation through Deep Reinforcement Learning: Fusing Multiple Visual Features and Reward Mechanisms

Contact Info

Product

Resources

About