Recurrent Convolutional Neural Network Regression for Continuous Pain Intensity Estimation in Video

Abstract

Automatic pain intensity estimation possesses a significant position in healthcare and medical field. Traditional static methods prefer to extract features from frames separately in a video, which would result in unstable changes and peaks among adjacent frames. To overcome this problem, we propose a real-time regression framework based on the recurrent convolutional neural network for automatic frame-level pain intensity estimation. Given vector sequences of AAM-warped facial images, we used a sliding-window strategy to obtain fixed-length input samples for the recurrent network. We then carefully design the architecture of the recurrent network to out- * Corresponding author put continuous-valued pain intensity. The proposed endto-end pain intensity regression framework can predict the pain intensity of each frame by considering a sufficiently large historical frames while limiting the scale of the parameters within the model. Our method achieves promising results regarding both accuracy and running speed on the published UNBC-McMaster Shoulder Pain Expression Archive Database.

Human Observer and Automatic Assessment of Movement Related Self-Efficacy in Chronic Pain: From Exercise to Functional Activity

“…Despite the face still receiving more attention than other affective channels in building affect aware systems (e.g., [32][33] [34]), the emergence of low-cost movement sensing technology is increasingly leading researchers to target movement as an affective modality, including in clinical contexts. Beyond work aimed at assessing affective states during sedentary clinical settings [35], there is also a growing interest in their assessment during physical activity and in situ [36].…”

Cumulative attributes for pain intensity estimation

“…Note that in comparing with [8], we report on their unprocessed results because our proposed approach does not include the additional person-specific normalization of the predictions which is implemented in [8]. Even though our PCC is slightly lower than Comparison of average error rates on higher pain levels for CA vs low level features [18] and [46] we achieve a 70% reduction in RMSE in comparison to [18] and 19% in comparison to [46] respectively. Note that, though a high PCC is desirable, it is also important to have a low prediction error rate.…”

“…Similar to [18], a recurrent convolutional network is presented in [46] which uses a time windowed flattened 1D frame features as input to the network. Egede et al [8] implemented a fusion of handcrafted and deep learned features with an additional person-specific normalisation of predictions.…”

Computational biology: deep learning

“…Al Rahhal et al [84] trained autoencoders to learn features from electrocardiogram signals and used them to detect various heartrelated disorders. As a completely different input, a video recording of a patient's face could be used to automatically estimate pain intensity with a recurrent convolutional neural network [85]. Just over the last year, there have been reports of applying convolutional neural networks in image-based diagnostics of age-related macular degeneration [86], diabetic retinopathy [87], breast cancer [88][89][90], brain tumors [91,92], cardiovascular disease [93], Alzheimer's disease [94], and many more diseases (Appendix to this article).…”