A Novel Technique for Size Constrained Video Storyboard Generation Using Statistical Run Test and Spanning Tree

Mohanta, Partha Pratim; Saha, Sanjoy Kumar; Chanda, Bhabatosh

doi:10.1142/s0219467813500010

Cited by 7 publications

(5 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Video sequence transitions are detected in [9] by tracking image changes, and selecting keyframes most similar to the average of all frames in shots. Shots selected by detecting video frame transition effects may not be well described using a single key-frame, and a statistical run test is used in [10] to segment shots into sub-shots before key-frame selection.…”

Section: Related Workmentioning

confidence: 99%

Rapid Probabilistic Interest Learning from Domain-Specific Pairwise Image Comparisons

Burke¹,

Mbonambi²,

Molala³

et al. 2017

Preprint

View full text Add to dashboard Cite

A great deal of work aims to discover general purpose models of image interest or memorability for visual search and information retrieval. This paper argues that image interest is often domain and user specific, and that mechanisms for learning about this domain-specific image interest as quickly as possible, while limiting the amount of data-labelling required, are often more useful to end-users. Specifically, this paper is concerned with the small to medium-sized data regime regularly faced by practising data scientists, who are often required to build turnkey models for end-users with domain-specific challenges. This work uses pairwise image comparisons to reduce the labelling burden on these users, and shows that Gaussian process smoothing in image feature space can be used to build probabilistic models of image interest extremely quickly for a wide range of problems, and performs similarly to recent deep learning approaches trained using pairwise ranking losses. The Gaussian process model used in this work interpolates image interest inferred using a Bayesian ranking approach over image features extracted using a pre-trained convolutional neural network. This probabilistic approach produces image interests paired with uncertainties that can be used to identify images for which additional labelling is required and measure inference convergence. Results obtained on five distinct datasets reinforce recent findings that pre-trained convolutional neural networks can be used to extract useful representations applicable across multiple domains, and highlight the fact that domain-specific image interest does not always correlate with concepts like image memorability.

show abstract

Section: Related Workmentioning

confidence: 99%

Rapid Probabilistic Interest Learning from Domain-Specific Pairwise Image Comparisons

Burke¹,

Mbonambi²,

Molala³

et al. 2017

Preprint

View full text Add to dashboard Cite

show abstract

“…To detect the scene boundaries, each shot is first represented by one or more representative frames (key-frames). For each shot, key-frame(s) are selected using the methodology presented in [8]. In this work, Wald-Wolfowitz run (WWR) test is applied on each shot to examine its consistency in terms of visual content.…”

Section: Shot and Key-frame Detectionmentioning

confidence: 99%

“…Our main contribution in this work are two folds: first, we have modified algorithm presented in [3] for scene detection; second, is to map the classical MST algorithm in a novel way to divide the key-frames of a scene into visually homogeneous groups and also to extract representative frame from each of these groups. The present endeavor is a continuation of our earlier work on shot segmentation [2], key-frame extraction [8] and scene segmentation [3]. However, care is taken to design the proposed algorithm such that any error committed in the earlier stages does not affect the output much.…”

Section: Introductionmentioning

confidence: 98%

“…Early approaches [6,7] considered first/last frame of the shot or both as representative. Mohanta et al [8] have proposed a scheme that decomposes a shot into subshots with uniform visual content and from each such sub-shot one frame is taken as the representative. Using these key-frames a shot level storyboard may be generated in [9].…”

Section: Introductionmentioning

confidence: 99%

“…Fuzzy-ART and Fuzzy C-Means based scheme [10] are also tried to minimize the redundancy. Mohanta et al [8] have presented a spanning tree based scheme to prepare a size constrained storyboard.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Static Summarization of Video Scenes Based on Minimal Spanning Tree

Mohanta

Chowdhury

Roy

et al. 2013

Lecture Notes in Computer Science

Self Cite

View full text Add to dashboard Cite

Abstract. The common practice for providing a static summarized view of a video is to create a storyboard. Storyboard is the chronological arrangement of the representative frames. Shot level storyboard suffers from redundancy as in a scene constituting shots normally repeat. Also the size of such storyboard is a constraint for many application. In this work we have considered scene as the more meaningful unit. We propose a state-based scene segmentation algorithm and also a minimal spanning tree based novel method to select the representative frames for the scenes. Storyboard consisting of scene level representative frames are much more compact than shot level storyboard. Moreover, scene being the semantic unit, flow of semantic content of the video data is well preserved. Experimental result confirms the claim.

show abstract