Automated detection of pain levels using deep feature extraction from shutter blinds-based dynamic-sized horizontal patches with facial images

Abstract

Pain intensity classification using facial images is a challenging problem in computer vision research. This work proposed a patch and transfer learning-based model to classify various pain intensities using facial images. The input facial images were segmented into dynamic-sized horizontal patches or “shutter blinds”. A lightweight deep network DarkNet19 pre-trained on ImageNet1K was used to generate deep features from the shutter blinds and the undivided resized segmented input facial image. The most discriminative features were selected from these deep features using iterative neighborhood component analysis, which were then fed to a standard shallow fine k-nearest neighbor classifier for classification using tenfold cross-validation. The proposed shutter blinds-based model was trained and tested on datasets derived from two public databases—University of Northern British Columbia-McMaster Shoulder Pain Expression Archive Database and Denver Intensity of Spontaneous Facial Action Database—which both comprised four pain intensity classes that had been labeled by human experts using validated facial action coding system methodology. Our shutter blinds-based classification model attained more than 95% overall accuracy rates on both datasets. The excellent performance suggests that the automated pain intensity classification model can be deployed to assist doctors in the non-verbal detection of pain using facial images in various situations (e.g., non-communicative patients or during surgery). This system can facilitate timely detection and management of pain.

Hybrid Techniques for the Diagnosis of Acute Lymphoblastic Leukemia Based on Fusion of CNN Features

“…ResNet50 takes the input images in the input layer, resizes the images to 224 × 224 pixels, and transfers them to the next layers. The model contains 49 convolutional layers with filters of different sizes to extract the features by wrapping the filter f ( t ) around the image x ( t ) as in Equation (4) [ 43 ]. The pooling layers receive huge amounts of neurons, which require complex operations that take a long time, and these layers work to reduce the high dimensions [ 44 ].…”

Using AI to Detect Pain through Facial Expressions: A Review

“…The principal outcomes differed among studies. For instance, one study focused only on the detection of pain [35], eight studies only on the estimation of multilevel pain intensity [36,37,39,41,42,44,45,48], and four studied both the detection of pain and the assessment of multilevel pain intensity [40,46,47,49]. Additionally, two studies proposed their automated detection model to differentiate between genuine and faked facial expressions of pain [38,43].…”

“…Four studies applied their automated pain detection systems to videos from their recruited patients [36,38,43,49], and eleven used them on at least one public database of pre-recorded patients experiencing pain [35,37,[39][40][41][42][44][45][46][47][48].…”

“…From the 11 studies using public databases, 7 used only one database [35,[39][40][41][42]46,47], while 3 used a second database to validate further their AI/ML model [37,45,48]. The most used was the UNBC-McMaster Shoulder Pain Archive database, utilized in 10 studies; this consisted of videos of 25 subjects with unilateral shoulder injuries whose pain was elicited by passive and active arm movements [35,37,[40][41][42][44][45][46][47][48]. Two studies used the MintPAIN database, consisting of videos of 20 participants with induced pain from electrical stimulation.…”

CASVM: An Efficient Deep Learning Image Classification Method Combined with SVM

“…Khairandish et al [15] and Gong et al [16] fused an SVM classifier in a neural network with robust classification accuracy. SVMs and neural networks are used in the fields of image classification and recognition, such as human biology [17][18][19], medical imaging [11,[20][21][22][23], and remote sensing [24,25], performing better at complex tasks than original neural network models.…”