A Sketch-Based Approach for Interactive Organization of Video Clips

Abstract

With the rapid growth of video resources, techniques for efficient organization of video clips are becoming appealing in the multimedia domain. In this article, a sketch-based approach is proposed to intuitively organize video clips by: (1) enhancing their narrations using sketch annotations and (2) structurizing the organization process by gesture-based free-form sketching on touch devices. There are two main contributions of this work. The first is a sketch graph, a novel representation for the narrative structure of video clips to facilitate content organization. The second is a method to perform context-aware sketch recommendation scalable to large video collections, enabling common users to easily organize sketch annotations. A prototype system integrating the proposed approach was evaluated on the basis of five different aspects concerning its performance and usability. Two sketch searching experiments showed that the proposed context-aware sketch recommendation outperforms, in terms of accuracy and scalability, two state-of-the-art sketch searching methods. Moreover, a user study showed that the sketch graph is consistently preferred over traditional representations such as keywords and keyframes. The second user study showed that the proposed approach is applicable in those scenarios where the video annotator and organizer were the same person. The third user study showed that, for video content organization, using sketch graph users took on average 1/3 less time than using a mass-market tool Movie Maker and took on average 1/4 less time than using a state-of-theart sketch alternative. These results demonstrated that the proposed sketch graph approach is a promising video organization tool.

VideoMap: An interactive and scalable visualization for exploring video content

“…We conducted a study to evaluate VideoMap, which demonstrated how the system can facilitate exploration of video content and significantly reduce browsing time needed to understand and find events of interest. Firstly, we compared VideoMap to two state-of-the-art video visualization and interaction methods: Storyline [22] and the Sketch Graph method [24].…”

“…Besides the traditional interaction method of using markers on a timeline to navigate through video content [23], new natural sketch-based interaction has been used in video authoring [12,24] by operating on a sketch summary. Visual feedback is also important for efficient interaction, following user preferences [25].…”

A new clothing image retrieval algorithm based on sketch component segmentation in mobile visual sensors

“…image, video, threedimensional (3D) model) has become an active research area. [22][23][24] Sketch-based retrieval methods allow users to submit a sketch drawing illustrating their search intent. Such a way of user query intent expression provides users much freedom to specify and clarify their search need.…”

“…Hu and Collomosse 6 adopted a Bag-of-Visual Words (BoVW) approach, incorporating Gradient Field HOG (GF-HOG) descriptor for SBIR. Liu et al 24 proposed an interactive sketch-based interface to organize video clips, enabling common users to easily draw simple sketches, and these sketches are automatically converted from keyframes of videos. Eitz et al 22 proposed a new sketch-based 3D model retrieval system based on bag-of-features framework.…”

A PMJ-inspired cognitive framework for natural scene categorization in line drawings

“…Due to less information stored in line drawings when compared to the color photographs, the usage of line drawings in intelligent process of visual media, including the retrieval and re-use of images [9,10], videos [11,12], 3D graphical models [13,14,15] and conceptual design in industry [16,17], has attracted considerable attention recently. The reader is referred to [5] for a survey.…”