Classification of summarized videos using hidden markov models on compressed chromaticity signatures

Abstract

As digital libraries and video databases grow, we need methods to assist us in the synthesis and analysis of digital video. Since the information in video databases can be measured in thousands of gigabytes of uncompressed data, tools for efficient summarizing and indexing of video sequences are indispensable. In this paper, we present a method for effective classification of different types of videos that makes use of video summarization that is the form of a storyboard of keyframes. To produce the summarization, we first generate a universal basis on which to project a video frame that effectively reduces any video to the same lighting conditions. Each frame is represented by a compressed chromaticity signature. We then set out a multi-stage hierarchical clustering method to efficiently summarize a video. Finally we classify TV videos using a trained hidden Markov model on the compressed chromaticity signatures and also temporal features of videos that are represented by their summaries.

Automatic Video Classification: A Survey of the Literature

“…Lu et al [64] avoid detecting shots altogether, instead identifying keyframes using clustering after first transforming frames to a chromatic color space to put all frames under the same lighting conditions. 4) Object-Based Features: Object-based features seem to be uncommon, perhaps because of the difficulty in detecting and identifying objects as well as the computational requirements to do so.…”

“…Truong et al [60] choose features they believe correspond to how humans identify genre. The features they use are average shot length, percentage of each type of shot transition (cut, fade, dissolve), camera movement, pixel luminance variance, rate of static scenes (i.e., little camera or object motion), [56] X X X Dimitrova et al [66] X X Truong et al [60] X X X X Kobla et al [7] X X Roach et al [75] X Roach et al [76] X X Pan and Faloutsos [77] X Lu et al [64] X Jadon et al [63] X X X Hauptmann et al [2] X X X Pan and Faloutsos [39] X Rasheed et al [62] X X Gibert et al [78] X X Yuan et al [65] X X X X Hong et al [79] X X X Brezeale and Cook [18] X Fischer et al [35] X X X X X Nam et al [4] X X X Huang et al [36] X X Qi et al [21] X Jasinschi and Louie [19] X X X X X Roach et al [42] X Rasheed and Shah [40] X X X Lin and Hauptmann [20] X Lee et al [37] X X Wang et al [13] X X X X Xu and Li [43] X X X Fan et al [8] X X X length of motion runs, standard deviation of a frame luminance histogram, percentage of pixels having brightness above some threshold, and percentage of pixels having saturation above some threshold. Classification is performed using the C4.5 decision tree to classify video into one of five classes: cartoon, commercial, music, news, or sports.…”

“…Lu et al [64] classify a video by first summarizing it. The color channel bands of each frame are normalized and then moved into a chromaticity color space.…”

On the use of computable features for film classification

“…The methods proposed in [20] and [30] are a few more examples which also work on compressed video data. Lu et al [17] applied an HMM based approach in the compressed domain and promising results were presented. Recently, the MPEG-7 community has focused on video indexing by using embedded semantic descriptors, [5].…”

Classification of video events using 4-dimensional time-compressed motion features

“…While effective in gesture and speech recognition, event modeling using graphical dynamic models including hidden Markov models of various flavors have tasted mixed success in modeling visual events. Use of hierarchical HMMs for combining modalities and modeling temporal video events includes [14,11,15,9]. Despite this application of dynamic graphical modeling, the performance for event modeling and detection continues to be a challenge in scenarios where a very large number of training samples are not available.…”