SOMHunter V2 at Video Browser Showdown 2021

Veselý, Patrik; Mejzlík, František; Lokoč, Jakub

doi:10.1007/978-3-030-67835-7_45

Cited by 11 publications

(8 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The similarity is then determined between a video and the two queries. SomHunter [15] uses a fixed number of three adjacent temporal queries. Visione also introduced temporal querying for the 2021 version of their retrieval system [1].…”

Section: Temporal Queriesmentioning

confidence: 99%

Multi-modal Interactive Video Retrieval with Temporal Queries

Heller

Arnold

Gasser

et al. 2022

MultiMedia Modeling

View full text Add to dashboard Cite

This paper presents the version of vitrivr participating at the Video Browser Showdown (VBS) 2022. vitrivr already supports a wide range of query modalities, such as color and semantic sketches, OCR, ASR and text embedding. In this paper, we briefly introduce the system, then describe our new approach to queries specifying temporal context, ideas for color-based sketches in a competitive retrieval setting and a novel approach to pose-based queries.

show abstract

Section: Temporal Queriesmentioning

confidence: 99%

Multi-modal Interactive Video Retrieval with Temporal Queries

Heller

Arnold

Gasser

et al. 2022

MultiMedia Modeling

View full text Add to dashboard Cite

show abstract

“…The version tested at LSC 2021 [6] builds on the previous LSC 2020 version and also comprises several approaches added for our VBS 2021 tools [13,22]. Specifically, a significant update is the option to position a text query and example image on a 2D canvas.…”

Section: Approaches Tested At Lsc 2021mentioning

confidence: 99%

“…Specifically, a significant update is the option to position a text query and example image on a 2D canvas. This feature allows searching in specific regions in image (e.g., just the upper left corner) either with arbitrary text sentences [22] or with example images [13] found at an external multimedia search server. Query localization is added for the price of higher storage costs (new feature vector for each region), but allows for more accurate and intuitive specification of the searched object position.…”

Section: Approaches Tested At Lsc 2021mentioning

confidence: 99%

“…Query localization is added for the price of higher storage costs (new feature vector for each region), but allows for more accurate and intuitive specification of the searched object position. The second feature available in the SOMHunter framework for VBS 2021 is an option to "relocate" parts of complex temporal queries [22]. The search scenario assumes that users start with the whole temporal query, giving usually more chances to find the target scene.…”

Section: Approaches Tested At Lsc 2021mentioning

confidence: 99%

“…For the LSC 2021 version, SOMHunter integrates both types of localized queries [13,22] guaranteeing rich universal options of text search initialization. Please note that collage queries can be initialized from an external image search engine supporting language localization, hence removing language barriers for arbitrary users.…”

Section: Approaches Tested At Lsc 2021mentioning

confidence: 99%

See 2 more Smart Citations

Enhanced SOMHunter for Known-item Search in Lifelog Data

Lokoč

Mejzlík

Veselý

et al. 2021

Proceedings of the 4th Annual on Lifelog Search Challenge

Self Cite

View full text Add to dashboard Cite

SOMHunter represents a modern light-weight framework for knownitem search in datasets of visual data like images or videos. The framework combines an effective W2VV++ text-to-image search approach, a traditional Bayesian like model for maintenance of relevance scores influenced by positive examples, and several types of exploration and exploitation displays. With this initial setting in 2020, already the first prototype of the system turned out to be highly competitive in comparison with other state-of-the-art systems at Video Browser Showdown and Lifelog Search Challenge competitions. In this paper, we present a new version of the system further extending the list of visual data search capabilities. The new version combines localized text queries with collage queries tested at VBS 2021 in two separate systems by our team. Furthermore, the new version of SOMHunter will integrate also the new CLIP text search model recently released by OpenAI. We believe that all the extensions will improve chances to effectively initialize the search that can continue with already supported browsing capabilities.

show abstract