3D Human Pose Estimation on a Configurable Bed from a Pressure Image

Abstract

Robots have the potential to assist people in bed, such as in healthcare settings, yet bedding materials like sheets and blankets can make observation of the human body difficult for robots. A pressure-sensing mat on a bed can provide pressure images that are relatively insensitive to bedding materials. However, prior work on estimating human pose from pressure images has been restricted to 2D pose estimates and flat beds. In this work, we present two convolutional neural networks to estimate the 3D joint positions of a person in a configurable bed from a single pressure image. The first network directly outputs 3D joint positions, while the second outputs a kinematic model that includes estimated joint angles and limb lengths. We evaluated our networks on data from 17 human participants with two bed configurations: supine and seated. Our networks achieved a mean joint position error of 77 mm when tested with data from people outside the training set, outperforming several baselines. We also present a simple mechanical model that provides insight into ambiguity associated with limbs raised off of the pressure mat, and demonstrate that Monte Carlo dropout can be used to estimate pose confidence in these situations. Finally, we provide a demonstration in which a mobile manipulator uses our network's estimated kinematic model to reach a location on a person's body in spite of the person being seated in a bed and covered by a blanket.

BodyPressure -- Inferring Body Pose and Contact Pressure from a Depth Image

“…Resting is characterized by a low degree of physical exertion, substantial contact with surrounding surfaces such as a bed or chair, and the fact that people spend an overwhelming portion of life resting. With the ability to learn complex mappings between images and labels using CNNs, researchers have inferred human resting pose from diverse human configurations, postures, and sensing modalities [2], [23], [24], [25], [26], [27], [28].…”

Longitudinal In-Bed Pressure Signals Decomposition and Gradients Analysis for Pressure Injury Monitoring

“…Other ANN approaches build networks from scratch or retraining the available networks with pressure data [ 20 , 23 ]. These approaches require a representative labelled training set, which can be difficult to obtain, especially if a second modality of data, such as optical videos [ 24 ] or motion capture (MoCap) data are needed [ 25 , 26 ]. Clever et al [ 26 ] proposed a system with two convolutional neural networks (CNN) to estimate the 3D joint positions of a person in a configurable bed setting.…”

“…These approaches require a representative labelled training set, which can be difficult to obtain, especially if a second modality of data, such as optical videos [ 24 ] or motion capture (MoCap) data are needed [ 25 , 26 ]. Clever et al [ 26 ] proposed a system with two convolutional neural networks (CNN) to estimate the 3D joint positions of a person in a configurable bed setting. To train the network, the authors collected MoCap information by connecting MoCap sensors to the body of the subjects.…”

Exploring a Fuzzy Rule Inferred ConvLSTM for Discovering and Adjusting the Optimal Posture of Patients with a Smart Medical Bed

“…SMBs have been seen in multiple forms, with various objectives being present in different implementations. i.e., measuring vital organs [ 3 ], 3D posture prediction with robot-assisted deep learning technology [ 4 ], and monitoring of respiratory rate [ 5 ].…”

“…The last approach identified in our literature search was the use of deep learning. Ulcer prevention was explored through ConvNets [ 4 ] and autoencoders [ 9 ], with the latter showing the highest accuracy when predicting four unique postures.…”