Toward Development and Evaluation of Pain Level-Rating Scale for Emergency Triage based on Vocal Characteristics and Facial Expressions

Abstract

In order to allocate the healthcare resource, triage classification system plays an important role in assessing the severity of illness of the boarding patient at emergency department. The self-report pain intensity numerical-rating scale (NRS) is one of the major modifiers of the current triage system based on the Taiwan Triage and Acuity Scale (TTAS). The validity and reliability of self-report scheme for pain level assessment is a major concern. In this study, we model the observed expressive behaviors, i.e., facial expressions and vocal characteristics, directly from audio-video recordings in order to measure pain level for patients during triage. This work demonstrates a feasible model, which achieves an accuracy of 72.3% and 51.6% in a binary and ternary pain intensity classification. Moreover, the study result reveals a significant association of current model and analgesic prescription/patient disposition after adjusted for patient-report NRS and triage vital signs.

Automatic Detection of Pain from Facial Expressions: A Survey

“…In this survey, we focused on automatic detection of pain from facial expressions. However, there have been efforts to apply machine learning methods to combine facial activity with other modalities such as vocalizations [28] or ECG, EMG, and skin conductance [29]. Improvements in pain recognition rates were reported when information from multiple modalities were fused.…”

“…Zhang et al [76] averaged the probabilities of selected pain-related AUs to calculate the pain intensity estimate. [126] statistical features from sequence of facial landmark distances and quadratic polynomial coefficients of mouth shape Tsai et al [28] bag of words from k-means based clusters of sequence of geometric features (facial landmark distances and quadratic polynomial coefficients of mouth shape)…”

“…Tsai et al [28] textural HOG from Three Orthogonal Planes (HOG-TOP) Chen et al [75] LBP from Three Orthogonal Planes (LBP-TOP) Kaltwang et al [113] combinations of LBP-TOP, LPQ-TOP, BSIF-TOP Yang et al [104] energy from optical flow Ghasemi et al [74] time-integral of histogram of oriented energies Irani et al [122], Irani et al [66] Hankel matrices from time series of Haar and/or Gabor Almost all classification and regression tasks were supervised. Ground truth in the form of pain or AU labels, and discrete or continuous-valued pain or AU intensities, were used to train the machine learning models.…”

“…Apart from facial expressions, attempts have been made to use other modalities either individually or in combination for automatic pain detection. For example, Aung et al [27] examined the use of body posture, body motion, and muscle activity for detecting patterns in body movements that could be indicative of pain during physical exercise; Tsai et al [28] combined facial expressions and acoustic features to detect pain intensities in emergency cases; Werner et al [29] investigated the use of facial expressions, electromyogram (EMG) recorded from trapezius muscles, and autonomic signals such as skin conductance and electrocardiogram (ECG) to automatically detect the different heat pain stimulus levels. Attempts have also been made to investigate the use of brain activation-acquired via either electroencephalography (EEG) [30] [31] or functional imaging [32]-for automatic pain assessment.…”

Two-Stream Attention Network for Pain Recognition from Video Sequences

“…Recent advances in both domains of computer vision and machine learning, combined with the release of several datasets designed specifically for pain-related research (e.g., UNBC-McMaster Shouder Pain Expression Archive Database [3], BioVid Heat Pain Database [4], Multimodal EmoPain Database [5] and SenseEmotion Database [6]), have fostered the development of a multitude of automatic pain assessment and classification approaches. These methods range from unimodal approaches, characterised by the optimisation of an inference model based on one unique and specific input signal (e.g., video sequences [7,8], audio signals [9,10] and bio-physiological signals [11][12][13]), to multimodal approaches that are characterised by the optimisation of an information fusion architecture based on parameters stemming from a set of distinctive input signals [14][15][16].…”