Seeing under the cover with a 3D U-Net: point cloud-based weight estimation of covered patients

Abstract

Purpose Body weight is a crucial parameter for patient-specific treatments, particularly in the context of proper drug dosage. Contactless weight estimation from visual sensor data constitutes a promising approach to overcome challenges arising in emergency situations. Machine learning-based methods have recently been shown to perform accurate weight estimation from point cloud data. The proposed methods, however, are designed for controlled conditions in terms of visibility and position of the patient, which limits their practical applicability. In this work, we aim to decouple accurate weight estimation from such specific conditions by predicting the weight of covered patients from voxelized point cloud data. Methods We propose a novel deep learning framework, which comprises two 3D CNN modules solving the given task in two separate steps. First, we train a 3D U-Net to virtually uncover the patient, i.e. to predict the patient’s volumetric surface without a cover. Second, the patient’s weight is predicted from this 3D volume by means of a 3D CNN architecture, which we optimized for weight regression. Results We evaluate our approach on a lying pose dataset (SLP) under two different cover conditions. The proposed framework considerably improves on the baseline model by up to $${16}{\%}$$ 16 % and reduces the gap between the accuracy of weight estimates for covered and uncovered patients by up to $${52}{\%}$$ 52 % . Conclusion We present a novel pipeline to estimate the weight of patients, which are covered by a blanket. Our approach relaxes the specific conditions that were required for accurate weight estimates by previous contactless methods and thus constitutes an important step towards fully automatic weight estimation in clinical practice.

MassNet: A Deep Learning Approach for Body Weight Extraction from A Single Pressure Image

“…Other researchers also focus on the possible occlusion issue. Alexander et al [8] studied the weight estimation of covered patients. They implemented a 3D U-Net to construct the 3D points clouds of the subject under blankets, and then a 3D CNN was applied for the weight regression.…”

How higher educational institutions cope with social change : the case of Tsinghua University, China

“…And a portion of approaches [8,56] utilize optical-flow or depth as their inputs which are least affected by domain shift compared with RGB. Bigalke et al [5] add human prior loss according to human anatomy. Kundu et al [29] define the uncertainty of predictions and control the uncertainty value while training.…”

“…To further ensure the anatomical plausibility of the pose, we introduce a human prior loss L prior 2 adapted from Bigalke et al [5]. Specifically, we formulate three losses to penalize asymmetric limb lengths L symm , implausible joint angles L angle , and implausible bone lengths L length .…”

“…We use the human prior loss similar to [5] to encourage the network to generate human-like 3D keypoints. The human prior loss is defined as three parts: 1) the predicting bone length should be in a reasonable range; 2) the predicting length of symmetric bones should be similar; 3) the predicting bone angles should be reasonable according to the kinematic of human.…”

“…The human prior loss is defined as three parts: 1) the predicting bone length should be in a reasonable range; 2) the predicting length of symmetric bones should be similar; 3) the predicting bone angles should be reasonable according to the kinematic of human. Different from Bigalke et al [5] who distribute each joint with a specific length range, we set one range for all bones. In our case, we set l min = 0.05m and l max = 0.7m.…”