Video-based Pain Level Assessment: Feature Selection and Inter-Subject Variability Modeling

Bourou, Dimitra; Pampouchidou, Anastasia; Tsiknakis, Manolis; Marias, Kostas; Simos, Panagiotis G.

doi:10.1109/tsp.2018.8441252

Cited by 5 publications

(6 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To address this, using the SAFEPA system, which is described in Figure 7 and operates on a frame-by-frame basis, we selected the highest-level prediction from all predictions made for the frames belonging to that video. The accuracy of the SAFEPA system on the entire unseen Biovid dataset was 33.28%, which is an improvement over the results reported in [37,38]. In [37], the authors proposed facial activity descriptors to detect pain and estimate its intensity.…”

Section: Experiments 3: Test In Real-time: How Well the Safepa System...mentioning

confidence: 83%

“…Their model was applied on the BioVid dataset and for leave-one-subject-out cross-validation; to recognize five levels of pain, an accuracy of 30.80% was achieved. In [38], Bourou and his colleagues proposed a feature selection and inter-subject variability modeling approach for video-based pain level assessment. They utilized lasso regression Furthermore, FEAPAS model was trained on JAFFE dataset using an ADAM optimizer and 150 × 150 input size and achieved a remarkable accuracy of 96.80% in 10-fold crossvalidation, which is a highly competitive result compared to the Ensemble classifier of VGG16 and ResNet50 models used in [17] that achieved an accuracy of 96.40%.…”

Section: Experiments 3: Test In Real-time: How Well the Safepa System...mentioning

confidence: 99%

“…Their model was applied on the BioVid dataset and for leave-one-subject-out cross-validation; to recognize five levels of pain, an accuracy of 30.80% was achieved. In [38], Bourou and his colleagues proposed a feature selection and inter-subject variability modeling approach for video-based pain level assessment. They utilized lasso regression to select the most informative features from the extracted geometric and color-based features, and then modeled inter-subject variability using a generalized linear mixed effects probit model.…”

Section: Experiments 3: Test In Real-time: How Well the Safepa System...mentioning

confidence: 99%

“…They utilized lasso regression to select the most informative features from the extracted geometric and color-based features, and then modeled inter-subject variability using a generalized linear mixed effects probit model. To evaluate their approach, the authors extracted a set of features from the facial regions of each video frame and trained a regression model to predict pain intensity [38]. They reported an accuracy of 27.13% based on 10-fold cross-validation [38].…”

Section: Experiments 3: Test In Real-time: How Well the Safepa System...mentioning

confidence: 99%

See 3 more Smart Citations

SAFEPA: An Expandable Multi-Pose Facial Expressions Pain Assessment Method

Alghamdi

Alaghband

2023

Applied Sciences

View full text Add to dashboard Cite

Accurately assessing the intensity of pain from facial expressions captured in videos is crucial for effective pain management and critical for a wide range of healthcare applications. However, in uncontrolled environments, detecting facial expressions from full left and right profiles remains a significant challenge, and even the most advanced models for recognizing pain levels based on facial expressions can suffer from declining performance. In this study, we present a novel model designed to overcome the challenges posed by full left and right profiles—Sparse Autoencoders for Facial Expressions-based Pain Assessment (SAFEPA). Our model utilizes Sparse Autoencoders (SAE) to reconstruct the upper part of the face from the input image, and feeds both the original image and the reconstructed upper face into two pre-trained concurrent and coupled Convolutional Neural Networks (CNNs). This approach gives more weight to the upper part of the face, resulting in superior recognition performance. Moreover, SAFEPA’s design leverages CNNs’ strengths while also accommodating variations in head poses, thus eliminating the need for face detection and upper-face extraction preprocessing steps needed in other models. SAFEPA achieves high accuracy in recognizing four levels of pain on the widely used UNBC-McMaster shoulder pain expression archive dataset. SAFEPA is extended for facial expression recognition, where we show it to outperform state-of-the-art models in recognizing seven facial expressions viewed from five different angles, including the challenging full left and right profiles, on the Karolinska Directed Emotional Faces (KDEF) dataset. Furthermore, the SAFEPA system is capable of processing BioVid Heat Pain datasets with an average processing time of 17.82 s per video (5 s in length), while maintaining a competitive accuracy compared to other state-of-the-art pain detection systems. This experiment demonstrates its applicability in real-life scenarios for monitoring systems. With SAFEPA, we have opened new possibilities for accurate pain assessment, even in challenging situations with varying head poses.

show abstract

Section: Experiments 3: Test In Real-time: How Well the Safepa System...mentioning

confidence: 83%

Section: Experiments 3: Test In Real-time: How Well the Safepa System...mentioning

confidence: 99%

“…Their model was applied on the BioVid dataset and for leave-one-subject-out cross-validation; to recognize five levels of pain, an accuracy of 30.80% was achieved. In [38], Bourou and his colleagues proposed a feature selection and inter-subject variability modeling approach for video-based pain level assessment. They utilized lasso regression to select the most informative features from the extracted geometric and color-based features, and then modeled inter-subject variability using a generalized linear mixed effects probit model.…”

Section: Experiments 3: Test In Real-time: How Well the Safepa System...mentioning

confidence: 99%

Section: Experiments 3: Test In Real-time: How Well the Safepa System...mentioning

confidence: 99%

See 2 more Smart Citations

SAFEPA: An Expandable Multi-Pose Facial Expressions Pain Assessment Method

Alghamdi

Alaghband

2023

Applied Sciences

View full text Add to dashboard Cite

show abstract

“…Werner et al [22] proposed a novel feature set, called facial activity descriptors, to describe facial actions for pain detection and pain intensity estimation. Bourou et al [23] calculated several distances (e.g., mean and median) from ROIs to classify pain expressions.…”

Section: Facial Expressionmentioning

confidence: 99%

Multimodal-Based Stream Integrated Neural Networks for Pain Assessment

Zhi

Zhou

et al. 2021

IEICE Trans. Inf. & Syst.

View full text Add to dashboard Cite

Pain is an essential physiological phenomenon of human beings. Accurate assessment of pain is important to develop proper treatment. Although self-report method is the gold standard in pain assessment, it is not applicable to individuals with communicative impairment. Nonverbal pain indicators such as pain related facial expressions and changes in physiological parameters could provide valuable insights for pain assessment. In this paper, we propose a multimodal-based Stream Integrated Neural Network with Different Frame Rates (SINN) that combines facial expression and biomedical signals for automatic pain assessment. The main contributions of this research are threefold. (1) There are four-stream inputs of the SINN for facial expression feature extraction. The variant facial features are integrated with biomedical features, and the joint features are utilized for pain assessment. (2) The dynamic facial features are learned in both implicit and explicit manners to better represent the facial changes that occur during pain experience. (3) Multiple modalities are utilized to identify various pain states, including facial expression and biomedical signals. The experiments are conducted on publicly available pain datasets, and the performance is compared with several deep learning models. The experimental results illustrate the superiority of the proposed model, and it achieves the highest accuracy of 68.2%, which is up to 5% higher than the basic deep learning models on pain assessment with binary classification.

show abstract

An Objective Pain Measurement Machine Learning Model through Facial Expressions and Physiological Signals

Zheng

Lin

2022

2022 28th International Conference on Mechatronics and Machine Vision in Practice (M2VIP)

View full text Add to dashboard Cite

Video-based Pain Level Assessment: Feature Selection and Inter-Subject Variability Modeling

Cited by 5 publications

References 26 publications

SAFEPA: An Expandable Multi-Pose Facial Expressions Pain Assessment Method

SAFEPA: An Expandable Multi-Pose Facial Expressions Pain Assessment Method

Multimodal-Based Stream Integrated Neural Networks for Pain Assessment

An Objective Pain Measurement Machine Learning Model through Facial Expressions and Physiological Signals

Contact Info

Product

Resources

About