Multimedia Analysis + Visual Analytics = Multimedia Analytics

2014 IEEE Conference on Visual Analytics Science and Technology (VAST)

2014

Abstract-The size and importance of visual multimedia collections grew rapidly over the last years, creating a need for sophisticated multimedia analytics systems enabling large-scale, interactive, and insightful analysis. These systems need to integrate the human's natural expertise in analyzing multimedia with the machine's ability to process large-scale data. The paper starts off with a comprehensive overview of representation, learning, and interaction techniques from both the human's and the machine's point of view. To this end, hundreds of references from the related disciplines (visual analytics, information visualization, computer vision, multimedia information retrieval) have been surveyed. Based on the survey, a novel general multimedia analytics model is synthesized. In the model, the need for semantic navigation of the collection is emphasized and multimedia analytics tasks are placed on the explorationsearch axis. The axis is composed of both exploration and search in a certain proportion which changes as the analyst progresses towards insight. Categorization is proposed as a suitable umbrella task realizing the exploration-search axis in the model. Finally, the pragmatic gap, defined as the difference between the tight machine categorization model and the flexible human categorization model is identified as a crucial multimedia analytics topic.Index Terms-Multimedia (image/video/music) visualization, machine learning.

Section: Introductionmentioning

confidence: 99%

Towards interactive, intelligent, and integrated multimedia analytics

2014 IEEE Conference on Visual Analytics Science and Technology (VAST)

2014

“…Methods combining content-based multimedia analysis of images and metadata with visual analytics [6] are still limited. One of the first systems doing so is the Informedia system which employs speech and video analysis in conjunction with effective user interfaces [7].…”

Section: A Multimedia Visualization and Analyticsmentioning

confidence: 99%

“…An example of a commercial system for factual and geospatial data is Tableau, building upon the query paradigm in [38]. Yet multimedia analytics, the combination of multimedia analysis and visual analytics [6] is just starting as a field and its potential still needs to be unlocked. Multimedia insight might pertain to a lot of different elements of the dataset.…”

Section: Introductionmentioning

confidence: 99%

Multimedia Pivot Tables for Multimedia Analytics on Image Collections

Koelma

2016

IEEE Trans. Multimedia

Abstract-We propose a multimedia analytics solution for getting insight in image collections by extending the powerful analytic capabilities of pivot tables, found in the ubiquitous spreadsheets, to multimedia. We formalize the concept of multimedia pivot tables and give design rules and methods for the multimodal summarization, structuring, and browsing of the collection based on these tables, all optimized to support an analyst in getting structural and conclusive insights. Our proposed solution provides truly interactive analytics on the visual content of image collections through concept detection results, as well as tags, geolocation, time and other metadata. We have performed user experiments with novice users on a dataset from Flickr to improve the initial design and with expert users in marketing and multimedia analysis on two domain specific datasets collected from Instagram. The results show that analysts are indeed capable of deriving structural and conclusive insights using the proposed multimedia analytics solution. On our website videos of the system in action are available.

Section: Multimedia Analyticsmentioning

confidence: 99%

“…This field, which combines visual analytics with multimedia analysis as depicted in Figure 1(a), has been developing over the last half decade [2]. The goal of multimedia analytics is to produce the processes, techniques and tools to allow users to efficiently and effectively analyze multimedia collections in order to gain insight and knowledge [2]. While multimedia analysis should be stressed to its limits to extract information automatically from the media files, visual analytics proposes an interactive process that must involve the user, through data selection and visualization techniques.…”

Section: Multimedia Analyticsmentioning

confidence: 99%

Ten Research Questions for Scalable Multimedia Analytics

Jónsson

et al. 2016

MultiMedia Modeling

Abstract. The scale and complexity of multimedia collections is ever increasing, as is the desire to harvest useful insight from the collections. To optimally support the complex quest for insight, multimedia analytics has emerged as a new research area that combines concepts and techniques from multimedia analysis and visual analytics into a single framework. State of the art multimedia analytics solutions are highly interactive and give users freedom in how they perform their analytics task, but they do not scale well. State of the art scalable database management solutions, on the other hand, are not yet designed for multimedia analytics workloads. In this position paper we therefore argue the need for research on scalable multimedia analytics, a new research area built on the three pillars of visual analytics, multimedia analysis and database management. We propose a specific goal for scalable multimedia analytics and present several important research questions that we believe must be addressed in order to achieve that goal.