The VISIONE Video Search System: Exploiting Off-the-Shelf Text Search Engines for Large-Scale Video Retrieval

Amato, Giuseppe; Bolettieri, Paolo; Carrara, Fabio; Debole, Franca; Falchi, Fabrizio; Gennaro, Claudio; Vadicamo, Lucia; Vairo, Claudio

doi:10.3390/jimaging7050076

Cited by 17 publications

(14 citation statements)

References 55 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Video search engines, such as [2,3,4] developed by the AIMH Lab [1], would benefit from sketches image analysis. Integrating the propose approach with them is a future work.…”

Section: Discussionmentioning

confidence: 99%

Sketches image analysis: Web image search engine usingLSH index and DNN InceptionV3

Schiavo,

Minutella,

Daole

et al. 2021

Preprint

View full text Add to dashboard Cite

The adoption of an appropriate approximate similarity search method is an essential prerequisite for developing a fast and efficient CBIR system, especially when dealing with large amount of data. In this study we implement a web image search engine on top of a Locality Sensitive Hashing (LSH) Index to allow fast similarity search on deep features. Specifically, we exploit transfer learning for deep features extraction from images. Firstly, we adopt InceptionV3 pretrained on ImageNet as features extractor, secondly, we try out several CNNs built on top of InceptionV3 as convolutional base fine-tuned on our dataset. In both of the previous cases we index the features extracted within our LSH index implementation so as to compare the retrieval performances with and without fine-tuning. In our approach we try out two different LSH implementations: the first one working with real number feature vectors and the second one with the binary transposed version of those vectors. Interestingly, we obtain the best performances when using the binary LSH, reaching almost the same result, in terms of mean average precision, obtained by performing sequential scan of the features, thus avoiding the bias introduced by the LSH index. Lastly, we carry out a performance analysis class by class in terms of recall against mAP highlighting, as expected, a strong positive correlation between the two.

show abstract

“…Video search engines, such as [2,3,4] developed by the AIMH Lab [1], would benefit from sketches image analysis. Integrating the propose approach with them is a future work.…”

Section: Discussionmentioning

confidence: 99%

Sketches image analysis: Web image search engine usingLSH index and DNN InceptionV3

Schiavo,

Minutella,

Daole

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…For color or semantic sketches, vitrivr supports a plethora of features [51,56], VERGE clusters to twelve predefined colors using the Color Layout MPEG-7 descriptor, and HTW uses a handcrafted low-level feature [18]. VIREO [42] and VISIONE [1] also support sketch search, with VISIONE extracting dominant colors with pretrained color hash tables [5,72] and objects using pretrained neural networks [47,48,78]. CollageHunter allows image collages, which enable localization of example image queries on a canvas.…”

Section: Image and Sketch Searchmentioning

confidence: 99%

“…In VISIONE, all modalities are mapped to text, which allows the usage of Apache Lucene as a search backend. Each modality is a sub-query and the Lucene QueryRescorer combines their search results [1]. In contrast, vitrivr uses a specialized database allowing vector, text and Boolean retrieval [15].…”

Section: Fusion Approachesmentioning

confidence: 99%

Interactive video retrieval evaluation at a distance: comparing sixteen interactive video search systems in a remote setting at the 10th Video Browser Showdown

Heller

Gsteiger

Bailer

et al. 2022

Int J Multimed Info Retr

Self Cite

View full text Add to dashboard Cite

The Video Browser Showdown addresses difficult video search challenges through an annual interactive evaluation campaign attracting research teams focusing on interactive video retrieval. The campaign aims to provide insights into the performance of participating interactive video retrieval systems, tested by selected search tasks on large video collections. For the first time in its ten year history, the Video Browser Showdown 2021 was organized in a fully remote setting and hosted a record number of sixteen scoring systems. In this paper, we describe the competition setting, tasks and results and give an overview of state-of-the-art methods used by the competing systems. By looking at query result logs provided by ten systems, we analyze differences in retrieval model performances and browsing times before a correct submission. Through advances in data gathering methodology and tools, we provide a comprehensive analysis of ad-hoc video search tasks, discuss results, task design and methodological challenges. We highlight that almost all top performing systems utilize some sort of joint embedding for text-image retrieval and enable specification of temporal context in queries for known-item search. Whereas a combination of these techniques drive the currently top performing systems, we identify several future challenges for interactive video search engines and the Video Browser Showdown competition itself.

show abstract

“…This demonstration paper presents the latest release of VI-SIONE [1,2,[4][5][6], an interactive large-scale video search system.…”

Section: Introductionmentioning

confidence: 99%

VISIONE: A Large-Scale Video Retrieval System with Advanced Search Functionalities

Amato

Bolettieri

Carrara

et al. 2023

Proceedings of the 2023 ACM International Conference on Multimedia Retrieval

Self Cite

View full text Add to dashboard Cite

VISIONE is a large-scale video retrieval system that integrates multiple search functionalities, including free text search, spatial color and object search, visual and semantic similarity search, and temporal search. The system leverages cutting-edge AI technology for visual analysis and advanced indexing techniques to ensure scalability. As demonstrated by its runner-up position in the 2023 Video Browser Showdown competition, VISIONE effectively integrates these capabilities to provide a comprehensive video retrieval solution. A system demo is available online, showcasing its capabilities on over 2300 hours of diverse video content (V3C1+V3C2 dataset) and 12 hours of highly redundant content (Marine dataset). The demo can be accessed at https://visione.isti.cnr.it/.

show abstract

The VISIONE Video Search System: Exploiting Off-the-Shelf Text Search Engines for Large-Scale Video Retrieval

Cited by 17 publications

References 55 publications

Sketches image analysis: Web image search engine usingLSH index and DNN InceptionV3

Sketches image analysis: Web image search engine usingLSH index and DNN InceptionV3

Interactive video retrieval evaluation at a distance: comparing sixteen interactive video search systems in a remote setting at the 10th Video Browser Showdown

VISIONE: A Large-Scale Video Retrieval System with Advanced Search Functionalities

Contact Info

Product

Resources

About