Multi-View Video Summarization

Abstract

Previous video summarization studies focused on monocular videos, and the results would not be good if they were applied to multi-view videos directly, due to problems such as the redundancy in multiple views. In this paper, we present a method for summarizing multi-view videos. We construct a spatio-temporal shot graph and formulate the summarization problem as a graph labeling task. The spatio-temporal shot graph is derived from a hypergraph, which encodes the correlations with different attributes among multi-view video shots in hyperedges. We then partition the shot graph and identify clusters of event-centered shots with similar contents via random walks. The summarization result is generated through solving a multi-objective optimization problem based on shot importance evaluated using a Gaussian entropy fusion scheme. Different summarization objectives, such as minimum summary length and maximum information coverage, can be accomplished in the framework. Moreover, multi-level summarization can be achieved easily by configuring the optimization parameters. We also propose the multi-view storyboard and event board for presenting multi-view summaries. The storyboard naturally reflects correlations among multi-view summarized shots that describe the same important event. The event-board serially assembles event-centered multi-view shots in temporal order. Single video summary which facilitates quick browsing of the summarized multi-view video can be easily generated based on the event board representation.Index Terms-Multi-objective optimization, multi-view video, random walks, spatio-temporal graph, video summarization.

Transductive Multi-view Embedding for Zero-Shot Recognition and Annotation

“…Learning multi-view embedding spaces Relating the low-level feature view and semantic views of data has been exploited in visual recognition and cross modal retrieval. Most existing work [33,13,16,36,10,9] focuses on modelling images/videos with associated text (e.g. tags on Flickr/YouTube).…”

Tensor-Based Shot Boundary Detection in Video Streams

“…Description of the main tasks in video abstraction is provided in the paper by Truong and Venkatesh [45]. On the other hand, Fu et al present an overview of the multiview video summarization methods [15]. Valdes and Martinez discuss on efficient video summarization and retrieval tools [47].…”

“…Finally, fds (C k (t?1) , R k , S k (t) ) from the step (16) in Algorithm 2, needs to be computed, but only once. However, it should be pointed out that the covariance matrix update rule (15), as well as (20), holds under assumption of the same mean value among data. If this is not true, then to keep the model accurate enough, also the mean updating component should be added to the aforementioned equations [34].…”

Untitled

“…Also the perimeter length L of the trapezoid is described as (8) Based on a similar analysis, the β rate of change of L simplifies to (9). It is clear from (9) that cameras positioned on the corner points of a trapezoid can be made to have equal stereoscopic distance when the trapezoid is half-hexagon in which all the sides are equal.…”

“…Therefore, it can be written that (10) The distance w can be obtained by equating the sum of the left and right hand-terms of (10) with (8). In this way, the coordinate description of the position of any camera located along AB can be easily obtained.…”