Aesthetics-Guided Summarization from Multiple User Generated Videos

Abstract

In recent years, with the rapid development of camera technology and portable devices, we have witnessed a flourish of user generated videos, which are gradually reshaping the traditional professional video oriented media market. The volume of user generated videos in repositories is increasing at a rapid rate. In today's video retrieval systems, a simple query will return many videos which seriously increase the viewing burden. To manage these video retrievals and provide viewers with an efficient way to browse, we introduce a system to automatically generate a summarization from multiple user generated videos and present their salience to viewers in an enjoyable manner. Among multiple consumer videos, we find their qualities to be highly diverse due to various factors such as a photographer's experience or environmental conditions at the time of capture. Such quality inspires us to include a video quality evaluation component into the video summarization since videos with poor qualities can seriously degrade the viewing experience. We first propose a probabilistic model to evaluate the aesthetic quality of each user generated video. This model compares the rich aesthetics information from several well-known photo databases with generic unlabeled consumer videos, under a human perception component indicating the correlation between a video and its constituting frames. Subjective studies were carried out with the results indicating that our method is reliable. Then a novel graph-based formulation is proposed for the multi-video summarization task. Desirable summarization criteria is incorporated as the graph attributes and the problem is solved through a dynamic programming framework. Comparisons with several state-of-the-art methods demonstrate that our algorithm performs better than other methods in generating a skimming video in preserving the essential scenes from the original multiple input videos, with smooth transitions among consecutive segments and appealing aesthetics overall.

Automatic Route Video Summarization based on Image Analysis for Intuitive Touristic Experience

“…There are many studies related to tourism. (1)(2)(3)(4)(5)(6)(7)(8)(9)(10)(11) In order to support tour planning, Kurata et al (1) and Hidaka et al (2) have proposed planning support systems. However, those systems only show a tour route on a map, and tourists cannot understand the tour route intuitively.…”

“…Various information such as texts, photos, maps, and videos collected in a whole city by various IoT devices are used for guiding, recommendation, and planning. (1)(2)(3)(4)(5)(6)(7)(8)(9)(10)(11) Tourists can watch many videos about sightseeing on the Internet through social networking services or YouTube. Videos are useful to plan a sightseeing tour because they include richer touristic information than the texts, maps, and photos in a tourism guide.…”

Untitled

“…Photography has not existed as long as other art forms, such as painting, music, and crafts, but it still appears pleasing or attractive to the eye and can inspire a person to convey a story or create an atmosphere [1,2]. A person may feel a chill through their body after viewing a winter snow scene, an invigorating "fresh" feeling from viewing a photo of a green tree or be eager to buy a product from viewing a commercial photo.…”

“…The state-of-the-art research paper proposed by Zhang utilized dependence analysis-based object decomposition to adjust objects' positions while keeping the original scene content [21]. As a conclusion of the composition-related approach, there were two remaining challenges: (1) inconsistency in the background when the foreground objects are moved; (2) interrelations among the complex foreground objects.…”

Ubiquitous collaborative multimedia capture of live experiences toward authoring extensible interactive multimedia documents

“…In the context of user-generated content, the growing relevance of multimedia content generated by amateur users motivates research related to models, methods, technologies and systems that support multimedia production -examples include support for the automatic summarization of multiple user-generated videos as reported by Zhang, Zhang and Zimmermann (2015), and for the automatic generation of photo collages as in the contribution by Bianco and Ciocca (2015).…”