Semi-supervised context adaptation: case study of audience excitement recognition

Abstract

To recognise just the same human reaction (for example, a strong excitement) in different contexts, customary behaviours in these contexts have to be taken into account; e.g. a happy sport audience may be cheering for long time, while a happy theatrical audience may produce only short bursts of laughter in order to not interrupt the performance. Tailoring recognition algorithms to contexts can be achieved by building either a context-specific or a generic system. The former is individually trained for each context to recognise sets of characteristic responses, whereas the latter-in contrast to the context-specific one-adapts to the context via significantly more lightweight modification of parameters. This paper follows the latter way and proposes a simple modification of a hidden Markov model (HMM) classifier that enables end users to adapt the generic system to a context or a personal perception of an annotator by labelling a fairly small number of data samples of each context. For better adaptability to the limited number of the user's annotations, the proposed semisupervised HMM classifier employs the maximum posterior marginal, rather than the more conventional maximum a posteriori decision rule. The proposed user-and contextadaptable semi-supervised HMM classifier was tested on recognising excitement of a show audience in three contexts (a concert hall, a circus, and a sport event), differing in how the excitement is expressed. In our experiments the proposed classifier recognised reactions of a non-neutral audience with 10% higher accuracy than the conventional HMM and support vector machine based classifiers.

Unobtrusive stress detection on the basis of smartphone usage data

“…In this case, the hidden states are to be recovered with the Bayesian MPM (maximum posterior marginal) rule that selects for each time moment the hidden state with the maximum posterior marginal probability. In this work, we employ HMM with MPM decision rule because it outperformed conventional HMM with MAP in two different studies comparing these two approaches: (1) detection of emotions of the show audience [32] and (2) detection of illnesses of the elderly [34].…”

Lightweight Adaptation of Classifiers to Users and Contexts: Trends of the Emerging Domain

“…(4) Cascaded training uses first unlabelled data for initial parameter estimates and then the labelled data for fine-tuning. This approach was applied to deep neural network [ 74 ], discrete HMM (hidden Markov model) [ 75 ], and MLP (multilayer perceptron) [ 26 ] based classifiers. The work [ 74 ] mainly aimed at increasing accuracy of offline training; difficulties to obtain the labelled data were not of main concern.…”

“…The work [ 26 ] reviewed approaches for reducing the need in labelled data, but also mainly for offline training. The work [ 75 ] was concerned with user-controlled runtime adaptation to indefinable situations (social behaviours) and therefore aimed at finding a quick and lightweight adaptation method. The main goal was to detect show highlights by recognising arousal of a show audience.…”

“…The evolutionary optimisation was used also in [ 75 ] for modifying a discrete HMM based on the MPM decisions. This work studied two approaches to adapt an affect recognition system: cascaded training (see Section 4.2.1 ) and model-level knowledge transfer.…”

“…Unlike the work [ 75 ], Caridakis et al [ 78 ] studied emotion recognition of individuals, and in their data contexts did not significantly differ from each other: all the data consisted of records of persons, communicating with four artificial computer characters [ 79 ]. The emotional expressions were not very intense because all the records were acquired in the same laboratory and communications with the artificial characters do not follow exactly the same social rules as with real humans [ 79 ].…”