Pain expression assessment based on a locality and identity aware network

Abstract

In clinical medicine, the pain feeling is a significant indicator for the medical condition of patients. Of late, automatic pain assessment methods have received more and more interests. Many researchers proposed corresponding methods and achieved impressive results. However, they always ignore the locality and individual differences of painful expression. Therefore, a locality and identity aware network (LIAN) for pain assessment is presented here. Concretely, for the locality characteristic, a locality aware module consisting of a twobranch structure, feature and attention branches, is presented. The former learns pain features by a deep network, while the latter guides the pain features to focus on the discriminational regions of pain. As for the individual differences, an identity aware module with a multi-task method is proposed to represent identity-related information to achieve identity-invariant pain assessment. Extensive experiments on public databases show the superiority of LIAN in pain assessment.

A Review of Automatic Pain Assessment from Facial Information Using Machine Learning

“…However, despite the promising pain assessment results gained by using the machine learning-based methods [1][2][3][4], most of them still need more improvement to reach satisfactory results. A very encouraging alternative is the use of deep learning models which have outperformed the machine learning models in pain assessment tasks [5][6][7][8][9][10][11][12][13].…”

“…The availability of many face images and/or video datasets for pain assessment has driven recent advances in the field of automatic pain assessment. The UNBC-McMaster Shoulder Pain Expression Archive Database (UNBC-McMaster) [44] is the one of the most widely utilized of these datasets [1][2][3][4][5][6][7][8][9][10][11][12][13][14][15][16][17][17][18][19][20][21]. This dataset was gathered from 25 adult participants suffering from shoulder pain, which form 48,398 RGB frames issued from 200 variable-length videos (see details in Table 1).…”

“…As a result, the UNBC-McMaster dataset size was reduced from 48,398 frames to 10,783 frames. Furthermore, another well-used dataset [6,8,20,45,59] is the BioVid Heat Pain dataset [45,59], which was formed by collecting 17300 RGB videos from 87 subjects of 5 s each with a frame rate of 25 fps. An inducted heat pain was the cause of the pain represented in these videos.…”

“…The conventional machine learning-based methods [1][2][3][4] manually extract handcrafted facial features, then classify them using conventional machine learning algorithms such as Support Vector Machines (SVMs) [47] and K-Nearest Neighbors (KNNs) [48]. However, deep learning-based methods [5][6][7][8][9][10][11][12][13] use deep learning models (e.g., Convolutional Neural Networks (CNNs) [49], Residual neural network (ResNet) [50], VGG [51] and Inception Network [52]) for both feature extraction and model-learning tasks. Recently, hybrid modelbased methods [14][15][16][17][18][19][20][21], which combine machine learning and/or deep learning have been proposed for the pain assessment task.…”

Multimodal automatic assessment of acute pain through facial videos and heart rate signals utilizing transformer-based architectures

“…They have also suggested a method of 3D distance computation among facial points, yielding comparable results. Patania et al ( 37 ) utilized deep graph neural network (GNN) architectures and dense maps of fiducial points to detect pain, while ( 38 ) presented a multi-task framework combining person identity recognition and pain level estimation, utilizing a CNN with an autoencoder attention module. Huang et al ( 39 ) detected facial regions and employed a multi-stream CNN for feature extraction, consisting of four sub-CNNs, one for each facial region.…”