Cross-modal Subspace Learning for Fine-grained Sketch-based Image Retrieval

Xu, Peng; Yin, Qiyue; Huang, Yongye; Song, Yi-Zhe; Ma, Zhanyu; Wang, Liang; Xiang, Tao; Kleijn, W. Bastiaan; Guo, Jun

doi:10.48550/arxiv.1705.09888

Cited by 1 publication

(1 citation statement)

References 65 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…), however, when describing the appearance or motion of those objects, the idea of keyword-based semantic descriptors fails. Other aspects that limit the scope of searching for video content include the level of annotation at the clip level, which does not consider the frames or even the objects located within the sequence of frames (3,4) . If the spatial and temporal resolutions could be calculated effectively, it might help search for accurate video content.…”

Section: Introductionmentioning

confidence: 99%

A Computational Meta-Learning Inspired Model for Sketch-based Video Retrieval

Pavithra¹,

Kumar²

2023

IJST

View full text Add to dashboard Cite

Objectives: To design and develop an efficient computing framework for sketch-based video retrieval using fine-grained intrinsic computational approach. Methods: The primary method of sketch-based video retrieval adopts multi-stream multi-modality of joint embedding method for improved P-SBVR from improved fine-grained KTH and TSF related dataset. It considers the potential aspects of the computation of significant visual intrinsic appearance details for sketch objects. The extracted appearance and motion-based features are used to train three different CNN baselines under strong and weak supervision. The system also implements a meta-learning model for different supervised settings to attain better performance of sketch-based video retrieval along with a relational module to overcome the problem of overfitting. Findings: The study derives specific sketch sequences from its formulated dataset to compute instance-level query processing for video retrieval. Further, it also addresses the limitations arising in the context of coarse-grained video retrieval models and sketch-based still image retrieval. The aggregated dataset for rich annotation assisted in the experimental simulation. The experimental evaluation with respect to the performance metric evaluates the 3D CNN baselines under strong supervision and weak-supervision where CNN BL-Type-2 attains maximum video retrieval accuracy of 99.96% for triplet grading feature under relational schema. CNN BL-Type-1 attains maximum retrieval accuracy of 97.40% considering the triplet grading features from the improved SBVR. The evaluation metric for the instance level retrieval process also considers true matching of sketches with the videos, it clearly shows that the appropriate appearance and motion based feature selection has enhanced the video retrieval accuracy up to 96.90% with 99.28% accuracy in action identification considering motion stream, 98.17% for appearance module and 98.45% for fusion module. Another important aspect of the proposed research context is that it addresses the problem of cross-modality while executing the simultaneous matching paradigm for visual appearances of the object with its movement appearing on particular video scenes. The experimental outcome shows https://www.indjst.org/

show abstract

Section: Introductionmentioning

confidence: 99%