3D Upper Body Reconstruction with Sparse Soft Sensors

Abstract

Three-dimensional (3D) reconstruction of human body has wide applications, for example, for customized design of clothes and digital avatar production. Existing vision-based systems for 3D body reconstruction require users to wear minimal or extreme-tight clothes in front of cameras, and thus suffer from privacy problems. In this work, we explore a novel solution based on a sparse number of soft sensors on a standard garment, and use it for capturing 3D upper body shape. We utilize the maximal stretching range by modeling the nonlinear performance profile for individual sensors. The body shape can be dynamically reconstructed by analyzing the relationship between mesh deformation and sensor reading, with a learning-based approach. The wearability and flexibility of our prototype allow its use in indoor/outdoor environments and for long-term breath monitoring. Our prototype has been extensively evaluated by multiple users with different body sizes and the same user for multiple days. The results show that our garment prototype is comfortable to wear, and achieves the state-of-the-art reconstruction performance with the advantages in privacy projection and application scenarios.

A Spring-Based Inductive Sensor for Soft and Flexible Robots

“…Thus, future work will go beyond the curvature control to estimate the pose of medical manipulators by fusing sensory information with the existing kinematic models. The proposed sensing solution is not only limited to medical robots, but may also be considered to extract the shape in other contexts like tensegrity mechanism control [49] or human body reconstruction [50]. In such cases, multiple sensors with machine learning or neural network methods might be adapted to tackle the complexity and the procedures established in [49], [50] could be followed.…”

Closed‐Loop Optimization of Soft Sensor Morphology Using 3D Printing of Electrically Conductive Hydrogel

“…Learning-based approaches can be used to model the nonlinear behavior of the sensor after installation. [14][15][16][17][18] However, the blackbox nature of these models makes them ineffective for any design purposes. A combination of body deformation models and simple sensor models has been used to optimize sensor morphologies, but the sim2real error gap is high in these cases.…”

Design and Development of a Hydrogel-based Soft Sensor for Multi-Axis Force Control

“…Hence, to model these soft strain sensors, the state estimation model must either use the time history of sensor states for future predictions [16] or use redundant sensor configurations to compensate for the nonlinearities [17]. Learning-based approaches using recurrent neural networks are typically used to directly incorporate the time dependent factors [18], [19], [20], [21]. Conversely, these time dependent hidden sensor states and external factors like temperature and humidity can be compensated using multiple sensors [17], [15].…”