Camera-based discomfort detection using multi-channel attention 3D-CNN for hospitalized infants

Abstract

Background: Detecting discomfort in infants is an important topic for their well-being and development.In this paper, we present an automatic and continuous video-based system for monitoring and detecting discomfort in infants.Methods: The proposed system employs a novel and efficient 3D convolutional neural network (CNN), which achieves an end-to-end solution without the conventional face detection and tracking steps. In the scheme of this study, we thoroughly investigate the video characteristics (e.g., intensity images and motion images) and CNN architectures (e.g., 2D and 3D) for infant discomfort detection. The realized improvements of the 3D-CNN are based on capturing both the motion and the facial expression information of the infants.Results: The performance of the system is assessed using videos recorded from 24 hospitalized infants by visualizing receiver operating characteristic (ROC) curves and measuring the values of area under the ROC curve (AUC). Additional performance metrics (labeling accuracy) are also calculated. Experimental resultsshow that the proposed system achieves an AUC of 0.99, while the overall labeling accuracy is 0.98. Conclusions:These results confirms the robustness by using the 3D-CNN for infant discomfort monitoring and capturing both motion and facial expressions simultaneously.

Assessment of Deep Learning-based Heart Rate Estimation using Remote Photoplethysmography under Different Illuminations

“…In the application of state monitoring, the algorithm should adapt to the light variation. For example, Sun et al [29] continuously monitored discomforts of infants over a long period, which requires the algorithm to work in complex light conditions. To estimate heart rate in extremely low light condition, Lin et al [30] proposed to use infrared spectrum to extract features.…”

Artificial Intelligence Based Pain Assessment Technology in Clinical Application of Real-World Neonatal Blood Sampling

“…Therefore, the overestimation of severe pain by our AI-NPA method is currently a purposeful choice driven by clinical needs. Although further improvement is required for our AI-NPA method, it is superior to those reported in the relevant literature [ 19 , 20 , 21 , 22 , 30 , 31 ], and especially in real-world scenarios. Nevertheless, the specificity of the automated NPA system for identifying severe pain in this study was completely acceptable in clinical practice.…”

“…Unfortunately, the current public reported neonatal pain-expression databases [ 15 , 16 , 17 , 18 ] still suffer from a limited number of newborns and samples, deficient population information, limited types of pain stimuli that have large differences with painful clinical procedures, the coarse labeling granularity of pain, etc. Moreover, these state-of-the-art databases and many AI-NPA methods [ 19 , 20 , 21 , 22 ] focus on ideal neonatal pain samples, which are samples with restrictions on the neonatal activities and facial posture in the data-collection stage, or with manual screening to avoid disturbed neonatal pain data. These methods make it difficult to meet the clinical requirements regarding accuracy, and they are not feasible for processing neonatal pain data collected in real-world clinical scenes.…”

Encoding the Photoreceptors of the Human Eye

“…Options could be different cat images, showing a partial image, blurred image (e.g., through the bushes), images with different luminance, the image at different rotational angles, and images with different backgrounds (e.g., residential, grassland, bushes, etc.) [31,32].…”

“…For example, approaching the eye, going away from the eye, moving in several directions, including walking, running, jumping, etc. [31,32].…”