Image-Based Pain Intensity Estimation Using Parallel CNNs with Regional Attention

Abstract

Automatic pain estimation plays an important role in the field of medicine and health. In the previous studies, most of the entire image frame was directly imported into the model. This operation can allow background differences to negatively affect the experimental results. To tackle this issue, we propose the parallel CNNs framework with regional attention for automatic pain intensity estimation at the frame level. This modified convolution neural network structure combines BlurPool methods to enhance translation invariance in network learning. The improved networks can focus on learning core regions while supplementing global information, thereby obtaining parallel feature information. The core regions are mainly based on the tradeoff between the weights of the channel attention modules and the spatial attention modules. Meanwhile, the background information of the non-core regions is shielded by the DropBlock algorithm. These steps enable the model to learn facial pain features adaptively, not limited to a single image pattern. The experimental result of our proposed model outperforms many state-of-the-art methods on the RMSE and PCC metrics when evaluated on the diverse pain levels of over 12,000 images provided by the publicly available UNBC dataset. The model accuracy rate has reached 95.11%. The experimental results show that the proposed method is highly efficient at extracting the facial features of pain and predicts pain levels with high accuracy.

AASTNet: An Attention-Aware Spatial-Temporal Network for Enhanced Pain Intensity Evaluation in Facial Image

“…Huang et al [6] utilized a VGG-16 network equipped with an attention mechanism to sequentially extract spatial features from video temporal sequences, which are then analyzed using a network composed of a GRU module and a temporal attention module for temporal feature extraction and pain intensity estimation. Ye et al [43] introduced a parallel CNN model for pain intensity evaluation, with one branch dedicated to extracting features in the core region through a balance between channel and spatial attention modules, whereas the other branch compensates for any missing information, facilitating the extraction of both global and local features for pain assessment.…”

A Review of Automatic Pain Assessment from Facial Information Using Machine Learning

“…The availability of many face images and/or video datasets for pain assessment has driven recent advances in the field of automatic pain assessment. The UNBC-McMaster Shoulder Pain Expression Archive Database (UNBC-McMaster) [44] is the one of the most widely utilized of these datasets [1][2][3][4][5][6][7][8][9][10][11][12][13][14][15][16][17][17][18][19][20][21]. This dataset was gathered from 25 adult participants suffering from shoulder pain, which form 48,398 RGB frames issued from 200 variable-length videos (see details in Table 1).…”

“…In recent years, numerous automatic pain estimation methods have been proposed . They aim at recognizing the pain level using different modalities such as facial expressions [11][12][13][14][15]19], voice [22], human behavior (e.g., human activity, body movement, coordination and speed) [22,23] and physiological signals (e.g., ECG brain signal and heart rate) [24][25][26]. Nonetheless, the study of facial expressions is the most often used data source to forecast pain assessment.…”

“…In the past, techniques for estimating pain intensity depended on manually extracting pain-related information with the assistance of medical professionals [27][28][29]. However, with the emergence of machine learning techniques and their success in the computer vision and image processing fields [30][31][32][33][34], many machine learning-based methods have tackled the task of pain assessment from facial expressions [11][12][13][14][15]19,22]. These methods have achieved impressive results.…”

“…However, deep learning-based methods [5][6][7][8][9][10][11][12][13] use deep learning models (e.g., Convolutional Neural Networks (CNNs) [49], Residual neural network (ResNet) [50], VGG [51] and Inception Network [52]) for both feature extraction and model-learning tasks. Recently, hybrid modelbased methods [14][15][16][17][18][19][20][21], which combine machine learning and/or deep learning have been proposed for the pain assessment task. These methods follow different ensemble learning strategies to leverage the advantages of different learning models.…”