Using Viewer's Facial Expression and Heart Rate for Sports Video Highlights Detection

Abstract

Viewer interest, evoked by video content, can potentially identify the highlights of the video. This paper explores the use of facial expressions (FE) and heart rate (HR) of viewers captured using camera and non-strapped sensor for identifying interesting video segments. The data from ten subjects with three videos showed that these signals are viewer dependent and not synchronized with the video contents. To address this issue, new algorithms are proposed to effectively combine FE and HR signals for identifying the time when viewer interest is potentially high. The results show that, compared with subjective annotation and match report highlights, 'non-neutral' FE and 'relatively higher and faster' HR is able to capture 60%-80% of goal, foul, and shot-ongoal soccer video events. FE is found to be more indicative than HR of viewer interest, but the fusion of these two modalities outperforms each of them.

Preliminary Simulation for Obtaining Accurate Heart Rate Data Using Imputation Methods

“…We found that there is no universal threshold value for classifying heart rate; researchers design their own standards based on their study. Considering recommendations from previous studies, we selected 45 to 130 beats per minute (bpm) as an acceptable range for our experimental simulation (Chakraborty et al, 2015; Chowdhury et al 2017). We tested the performances of linear interpolation, Kalman, spline and Stineman interpolation.…”

Synchronous prediction of arousal and valence using LSTM network for affective video content analysis

“…color, texture, and motion), or emotional responses from viewers during watching data stimuli (e.g. facial expression [8], EEG signal [9], heart rate [10], and multiple physiological signals [11]). In this paper, we focus on approaches belonging to the first way that represents affective dimensions using features extracted from the data.…”

Automatic classification of physical exercises from wearable sensors using small dataset from non-laboratory settings

“…We computed some overall time domain features like mean and standard deviation for each activity using all heart rate samples for that activity. Adopting a similar approach [14], for each activity, our work also calculated several timevarying features of heart-rate such as derivatives, gradient, energy, and variance. Obtained heart rate data was time stamped in beats-per-minute and a sliding window (with a span of 9 seconds) was used to compute the above temporal features.…”