Personalized video summarization using sift

Abstract

In this paper, we propose a methodology for generating users' tailored video abstracts. First, video frames are scored by a group of video experts (operators) according to audio, visual and textual content of the video. Then, a matrix that contains the relevancy scores of each video scene into a number of pre-defined categories is computed using Scaled Invariant Feature Transform (SIFT) features, which are computed pairwise for representative keyframes and delegate images from the training collection. Later, for profiling purposes, an end-user's interest levels towards those high-level visual concepts (categories) are captured in the form of a vector. As a result of combining these two groups of data, the user's priorities in regards to different video segments can be determined. In the next stage, the initial averaged scores of the frames are updated based on the end-users' generated profiles. Eventually, the highest scored video frames alongside the auditory and textual content are inserted into final digest. The effectiveness of this approach has been evaluated by comparing the video summaries generated by this system against the results from a number of automatic and semi-automatic summarization tools that use different modalities for abstraction.

The influence of human factors on 360∘ mulsemedia QoE

“…The significance of human factors has evolved with the proliferation of multi-user information systems as well as the diversity of services they provide. Today, the pursuit of adapting and personalizing web-based systems is a common phenomenon in areas such as e-commerce and e-learning [44,45,46,47], to name but the most popular.…”

Video Skimming: Taxonomy and Comprehensive Survey

“…Emphasis has been given to consider most publications in reputed journals and conferences 1 over the past decade as the last generic survey on video summarization [168] appeared in the year 2007. (Best viewed in color) in terms of the kind of frames [27], [56] the viewer likes to see in the skim. Personalization can be done non-intrusively by observing viewer behavior [190] or by learning from viewer comments [22].…”

“…Camera movement based semantics can be determined [17] for sports summarization. [74] identifies a variety of concepts such as a beach, people, flowers, indoors, etc., for ranking the frames and [27] provides for personalization by adapting summary to viewer preferences of concepts. The works in [16,98,111,169] not only identify the concepts but also model the interactions between the prominent entities (any predefined person or object) for determining the representative ones.…”

“…[178] considers learning the semantic concepts using a classifier and includes quality (using motion) information for determining important segments. [27] provides personalization by maintaining user profile that contains concept preferences. [109] preserves the semantic meaning/content of the video with reduced storage space and is capable of reconstructing the whole video.…”

Design and evaluation of a video retrieval environment for exploratory searches in a learning context

“…A personalized video summarization framework used clustered users' reactions as training data to produce semantic preference in the creation of personalized summaries (Chung, Hsiung, Wei, & Lee, 2014). Darabi and Ghinea (2015) proposed video summaries to be personalized by adapting the summary to the viewer's profile that contained the concept preference. These work generated users-tailored video summaries based on the high-level human interpretable concepts.…”