A hidden Markov model framework for video segmentation using audio and image features

Boreczky, John; Wilcox, Lynn

doi:10.1109/icassp.1998.679697

Cited by 150 publications

(90 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…After a learning step using the Baum-Welch algorithm, segmentation is performed using the Viterbi algorithm. A similar approach using Hidden Markov Models has been proposed by Boreczky in [15]. The model is based on image, audio, and motion features.…”

Section: Hidden Markov Modelsmentioning

confidence: 99%

A review of real-time segmentation of uncompressed video sequences for content-based search and retrieval

2003

View full text Add to dashboard Cite

We present in this paper a review of methods for segmentation of uncompressed video sequences. Video segmentation is usually performed in the temporal domain by shot change detection. In case of real-time segmentation, computational complexity is one of the criteria which has to be taken into account when comparing different methods. When dealing with uncompressed video sequences, this criterion is even more significant. However previous published reviews did not involve complexity criterion when comparing shot change detection methods. Only recognition rate and ability to classify detected shot changes were considered. So contrary to previous reviews we give here complexity of most of the described methods. We review in this paper an extensive set of methods presented in the literature and classify them in several parts, depending on the information used to detect shot changes. The earliest methods were comparing successive frames by relying on most simple elements, that is to say pixels. Comparison could be performed on a global level, so methods based on histograms were also proposed. Block-based methods have been considered to process data at an intermediate level, between local (using pixels) and global (using histograms) levels. More complex features can be involved, resulting in feature-based methods. Alternatively some methods rely on motion as a criterion to detect shot changes. Finally different kinds of information could be combined together in order to increase the quality of shot change detection. So our review will detail segmentation methods

show abstract

Section: Hidden Markov Modelsmentioning

confidence: 99%

A review of real-time segmentation of uncompressed video sequences for content-based search and retrieval

2003

View full text Add to dashboard Cite

show abstract

“…HMMs have been used with success for numerous sequence recognition tasks, including speech recognition [20], video segmentation [5], sports event recognition [25], and broadcast news segmentation [11]. HMMs introduce a state variable q t and factor the joint distribution of the observation sequence and the underlying (unobserved) HMM state sequence into two simpler distributions, namely emission distributions p(o t |q t ) and transition distributions p(q t |q t−1 ).…”

Section: Notation and Modelsmentioning

confidence: 99%

Multi Channel Sequence Processing

Bengio

Bourlard

2005

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Abstract. This paper summarizes some of the current research challenges arising from multichannel sequence processing. Indeed, multiple real life applications involve simultaneous recording and analysis of multiple information sources, which may be asynchronous, have different frame rates, exhibit different stationarity properties, and carry complementary (or correlated) information. Some of these problems can already be tackled by one of the many statistical approaches towards sequence modeling. However, several challenging research issues are still open, such as taking into account asynchrony and correlation between several feature streams, or handling the underlying growing complexity. In this framework, we discuss here two novel approaches, which recently started to be investigated with success in the context of large multimodal problems. These include the asynchronous HMM, providing a principled approach towards the processing of multiple feature streams, and the layered HMM approach, providing a good formalism for decomposing large and complex (multi-stream) problems into layered architectures. As briefly reported here, combination of these two approaches yielded successful results on several multi-channel tasks, ranging from audio-visual speech recognition to automatic meeting analysis. 2 IDIAP-RR 05-04

show abstract

“…Dynamic graph models, and in particular Dynamic Bayesian Networks (DBNs) including Hidden Markov Models (HMMs) and their variants, have become increasingly popular for modelling and analysing space-time visual data [8,13,4,7,12,9]. By using a DBN, we assume that dynamic visual data are generated sequentially by some hidden states of a dynamic scene evolving over time.…”

Section: Introductionmentioning

confidence: 99%

Optimal Dynamic Graphs for Video Content Analysis

Xiang¹,

Gong²

2006

Procedings of the British Machine Vision Conference 2006

View full text Add to dashboard Cite

This study addresses the problem of learning the optimal structure of a dynamic graphical model for video content analysis given sparse data. We propose a Completed Likelihood AIC (CL-AIC) scoring function that differs from existing ones by optimising explicitly both the explanation and prediction capabilities of a model simultaneously. We demonstrate that CL-AIC is superior to existing scoring functions including BIC, AIC and ICL in building dynamic graph models for video content analysis.

show abstract

A hidden Markov model framework for video segmentation using audio and image features

Cited by 150 publications

References 12 publications

A review of real-time segmentation of uncompressed video sequences for content-based search and retrieval

A review of real-time segmentation of uncompressed video sequences for content-based search and retrieval

Multi Channel Sequence Processing

Optimal Dynamic Graphs for Video Content Analysis

Contact Info

Product

Resources

About