ICLIC: Interactive categorization of large image collections

Corput, Paul van der; Wijk, Jarke J. van

doi:10.1109/pacificvis.2016.7465263

Cited by 11 publications

(5 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…If we consider low-level tasks by Amar et al [1], only a few of the 10 tasks can be applied to images [66]. Thus, an important challenge in interactive visualization of image data is automatic extraction of semantic information, interactive exploration of categories, or both [53,62,67].…”

Section: Image Browsingmentioning

confidence: 99%

DendroMap: Visual Exploration of Large-Scale Image Datasets for Machine Learning with Treemaps

Bertucci¹,

Hamid²,

Anand³

et al. 2022

Preprint

View full text Add to dashboard Cite

Fig. 1. With DendroMap, users can explore large-scale image datasets by overviewing the overall distributions and zooming down into hierarchies of image groups at multiple levels of abstraction. In this example, we visualize images of the CIFAR-100 dataset by hierarchically clustering the image representations obtained from a ResNet50 image classification model. (B) DendroMap View displays these clusters of images organized as a hierarchical structure by adapting Treemaps. By clicking on a cluster, a user can interactively (C) Zoom into that image group, revealing subgroups that replace and fill the available space with animation (see the submitted video).The user clicked on a cluster for organism images, which creates distinct subgroups of fish, insects, worms, fruits, and flowers. With (A) Sidebar View, the user can dynamically adjust the number of clusters to be displayed and inspect the class-level statistics.

show abstract

Section: Image Browsingmentioning

confidence: 99%

DendroMap: Visual Exploration of Large-Scale Image Datasets for Machine Learning with Treemaps

Bertucci¹,

Hamid²,

Anand³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…The features (esp. semantically meaningful ones, such as concept labels) can be used as additional metadata (e.g., [38]) or to build a content-based index to fuel search capabilities. Indexing approaches include clustering-based approaches such as product quantization [17] or extended cluster pruning [13], and hash-based approaches [2], especially those based on locality-sensitive hashing [6].…”

Section: Related Workmentioning

confidence: 99%

“…There are approaches that incorporate interactive model building to cover a wider range of the exploration-search axis. To advance the analytic session, they usually make use of a rich set of filters on the data (e.g., [3,19,20,38,39]), an interactive (multimodal) learning model (e.g., [15,43]), or a combination of both. Whilst these techniques go beyond mere search, on the exploration-search axis, they tend to lean towards search anyway: they simply fetch what the users are looking for or what they found relevant previously.…”

Section: Introductionmentioning

confidence: 99%

II-20: Intelligent and pragmatic analytic categorization of image collections

Zahálka¹,

Worring²,

Wijk³

2020

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Multimedia analytics systems, such as Multimedia Pivot Tables [31], ICLIC [28], and Blackthorn [38] facilitate search and exploration in large collections of multimedia data as well as interactive multimodal learning. Blackthorn, for example, compresses semantic information from the visual and text domain and learns user preferences on the fly from the interactions with the system in a relevance feedback framework.…”

Section: Multimedia Analyticsmentioning

confidence: 99%

Interactive Search and Exploration in Discussion Forums Using Multimodal Embeddings

Gornishka

Rudinac

Worring

2019

MultiMedia Modeling

View full text Add to dashboard Cite

In this paper we present a novel interactive multimodal learning system, which facilitates search and exploration in large networks of social multimedia users. It allows the analyst to identify and select users of interest, and to find similar users in an interactive learning setting. Our approach is based on novel multimodal representations of users, words and concepts, which we simultaneously learn by deploying a general-purpose neural embedding model. We show these representations to be useful not only for categorizing users, but also for automatically generating user and community profiles. Inspired by traditional summarization approaches, we create the profiles by selecting diverse and representative content from all available modalities, i.e. the text, image and user modality. The usefulness of the approach is evaluated using artificial actors, which simulate user behavior in a relevance feedback scenario. Multiple experiments were conducted in order to evaluate the quality of our multimodal representations, to compare different embedding strategies, and to determine the importance of different modalities. We demonstrate the capabilities of the proposed approach on two different multimedia collections originating from the violent online extremism forum Stormfront and the microblogging platform Twitter, which are particularly interesting due to the high semantic level of the discussions they feature. CCS CONCEPTS• Information systems → Multimedia and multimodal retrieval. KEYWORDS multimedia analytics, search, exploration, interactive learning, multimodal embeddings, online discussion forums, social multimedia • First, compact but meaningful multimodal content representations are needed to ensure the interactivity of the system

show abstract

ICLIC: Interactive categorization of large image collections

Cited by 11 publications

References 22 publications

DendroMap: Visual Exploration of Large-Scale Image Datasets for Machine Learning with Treemaps

DendroMap: Visual Exploration of Large-Scale Image Datasets for Machine Learning with Treemaps

II-20: Intelligent and pragmatic analytic categorization of image collections

Interactive Search and Exploration in Discussion Forums Using Multimodal Embeddings

Contact Info

Product

Resources

About