Recent developments in openSMILE, the munich open-source multimedia feature extractor

Abstract

We present recent developments in the openSMILE feature extraction toolkit. Version 2.0 now unites feature extraction paradigms from speech, music, and general sound events with basic video features for multi-modal processing. Descriptors from audio and video can be processed jointly in a single framework allowing for time synchronization of parameters, on-line incremental processing as well as off-line and batch processing, and the extraction of statistical functionals (feature summaries), such as moments, peaks, regression parameters, etc. Postprocessing of the features includes statistical classifiers such as support vector machine models or file export for popular toolkits such as Weka or HTK. Available low-level descriptors include popular speech, music and video features including Mel-frequency and similar cepstral and spectral coefficients, Chroma, CENS, auditory model based loudness, voice quality, local binary pattern, color, and optical flow histograms. Besides, voice activity detection, pitch tracking and face detection are supported. openSMILE is implemented in C++, using standard open source libraries for on-line audio and video input. It is fast, runs on Unix and Windows platforms, and has a modular, component based architecture which makes extensions via plug-ins easy. openSMILE 2.0 is distributed under a research license and can be downloaded from http://opensmile.sourceforge.net/.

Acoustic-Prosodic Automatic Personality Trait Assessment for Adults and Children

“…In our experiments, we used two sets of features extracted with openSMILE [6], and a set of knowledge-oriented features known in the literature to have impact on the personality classification tasks tackled here, henceforth referred to as knowledge-based features. The Interspeech 2012 Speaker Trait ChallengePersonality Sub-challenge feature set consists of 6125 features, and has been used in the present work to provide a set of baseline results (IS2012).…”

A Survey on perceived speaker traits: Personality, likability, pathology, and the first challenge

“…Again, we used TUM's open-source openSMILE feature extractor (Eyben et al, 2010(Eyben et al, , 2013Eyben, 2014) and provided extracted feature sets on a per-chunk level and a configuration file to allow for additional frame-level feature extraction. The general strategy was to preserve the high-dimensional 2011 SSC feature set including energy, spectral, and voicing related low-level descriptors (LLDs); a few LLDs were added including logarithmic HNR, spectral harmonicity, and psycho-acoustic spectral sharpness, as in the AVEC 2011 set.…”

“…The latest version of openSMILE is described in Eyben et al (2013). Details can be found in the openSMILE handbook, source code, and configuration file 2 , and in Eyben (2014).…”

Efficient acoustic detector of gunshots and glass breaking

“…In a similar way, accumulated acoustic score is calculated as Mahalanobis distance of the input feature vector to the PDFs (b n t ) with incoming feature vector according to the input symbol n on the same transition. These accumulated scores are used together for comparing tokens to each other according to Viterbi decoding criterion (5). The result (sequence of acoustic events W ) is then defined as sequence of states Q (path through the search network) with maximum probability max Q P (O, Q|W, a ) when a set of O feature vectors is observed and an acoustic model a is used.…”

“…This kind of system can be built using any available toolkit for processing input audio signal [5] and toolkit for classification [27] or simply using an all-in-one system, for example a speech recognition engine [12,13]. These systems, although they are capable of classification of the acoustic events if their acoustic models are provided, tend to be unnecessarily complicated containing redundant functions that slow down the whole system [20].…”