Seeing Under the Cover: A Physics Guided Learning Approach for In-bed Pose Estimation

Abstract

Human in-bed pose estimation has huge practical values in medical and healthcare applications yet still mainly relies on expensive pressure mapping (PM) solutions. In this paper, we introduce our novel physics inspired visionbased approach that addresses the challenging issues associated with the in-bed pose estimation problem including monitoring a fully covered person in complete darkness. We reformulated this problem using our proposed Under the Cover Imaging via Thermal Diffusion (UCITD) method to capture the high resolution pose information of the body even when it is fully covered by using a long wavelength IR technique. We proposed a physical hyperparameter concept through which we achieved high quality groundtruth pose labels in different modalities. A fully annotated in-bed pose dataset called Simultaneously-collected multimodal Lying Pose (SLP) is also formed/released with the same order of magnitude as most existing large-scale human pose datasets to support complex models' training and evaluation. A network trained from scratch on it and tested on two diverse settings, one in a living room and the other in a hospital room showed pose estimation performance of 98.0% and 96.0% in PCK0.2 standard, respectively. Moreover, in a multi-factor comparison with a state-of-the art in-bed pose monitoring solution based on PM, our solution showed significant superiority in all practical aspects by being 60 times cheaper, 300 times smaller, while having higher pose recognition granularity and accuracy.

Review of Artificial Intelligence Techniques in Imaging Data Acquisition, Segmentation, and Diagnosis for COVID-19

“…Scan range can be identified by detecting anatomical joints of the subject from the images. Much recent work [27][28][29] has focused on estimating the 2D [30][31][32][33][34][35] or 3D keypoint locations [28,[36][37][38][39] on the patient body. These keypoint locations usually include major joints such as the neck, shoulders, elbows, ankles, wrists, and knees.…”

Graph-Based Deep Learning for Medical Diagnosis and Analysis: Past, Present and Future

“…As a result of this focus, the areas of patient behaviour assessment have received less attention. While several in-clinic systems using CNN and RNN-based models have been introduced to enable comprehensive data analysis through accurate and granular quantification of a patient’s movements [ 231 , 232 , 233 ], these methods are not yet sufficiently accurate for widespread clinical use, yet we argue that graph neural networks have a great potential in these application areas.…”

Towards Contactless Patient Positioning

“…For all RGB and thermal experiments, we use the SLP [9] dataset, which comes with images of people lying on a bed and covered by a cloth under varying cover conditions: no cover (uncover), "light" cover (referred to as cover1), and "heavy" cover (cover2). For all RGB and depth experiments, we use the publicly available CAD [21] and PKU [22] datasets, along with a proprietary medical scan patient setup (SCAN) dataset.…”

“…Much recent algorithmic work [7]- [9] in patient positioning has focused on estimating the 2D or 3D keypoint locations on the patient body. Such keypoints represent only a very sparse sampling of the full body mesh in the 3D space that defines the digital human body.…”