A two level strategy for audio segmentation

Abstract

International audienceIn this paper we are dealing with audio segmentation. The audio tracks are sampled in short sequences which are classified into several classes. Every sequence can then be further analyzed depending on the class it belongs to. We first describe simple techniques for segmentation in two or three classes. These methods rely on amplitude, spectral or cepstral analysis, and classical Hidden Markov Models. From the limitations of these approaches, we propose a two level segmentation process. The segmentation is performed by computing several features for each audio sequence. These features are computed either on a complete audio segment or on a frame (set of samples) which is a subset of the audio segment. The proposed approach for microsegmentation of audio data consists of a combination of a K-mean classifier at the segment level and of a Multidimensional Hidden Markov Model system using the frame decomposition of the signal. A first classification is obtained using the K-mean classifier and segment-based features. Then final result comes from the use of Multidimensional Hidden Markov Models and frame-based features involving temporary results. Multidimensional Hidden Markov Models are an extension of classical Hidden Markov Models dedicated to multicomponent data. They are particularly adapted to our case where each audio segment can be characterized by several features of different natures. We illustrate our methods in the context of analysis of football audio tracks

TimeSeriesClustering: An extensible framework in Julia

“…This task occurs in many disciplines, for example in finding repeated animal behavior (Mueen, Keogh, Zhu, Cash, & Westover, 2013), finding regulatory elements in DNA (Das & Dai, 2007), and finding patterns in EEG signals (Castro & Azevedo, 2010). Another application area of our software is segmentation and clustering of audio datasets (Siegler, Jain, Raj, & Stern, 1997, Lefèvre & Vincent (2011, Kamper, Livescu, & Goldwater (2017)).…”

An Efficient Approach for Segmentation, Feature Extraction and Classification of Audio Signals

“…Initial classification was performed by using the k-means classifier and segment-related features. Final classification was done by using the Multidimensional Hidden Markov Models and frame-related features [11].…”

Audio segmentation of broadcast news in the Albayzin-2010 evaluation: overview, results, and discussion

“…The latter evaluation showed that the overlapping segments accounted for more than 70% of errors produced by every submitted system. Despite the interest shown in mixed sound detection in the recent years [11][12][13], it still remains a challenging problem.…”

“…The most common procedure for doing that is by Viterbi decoding, i.e., using a dynamic programming algorithm to find in a recursive a manner the most probable sequence of HMM states. The HMM-based audio segmentation approach borrowed from speech/ speaker recognition applications has been successfully applied in [4,10,13] and many other studies.…”