Deformation Capture via Soft and Stretchable Sensor Arrays

Abstract

Fig. 1. Left to right: We propose a method for the fabrication of soft and stretchable silicone based capacitive sensor arrays. The sensor provides dense stretch measurements that, together with a data-driven prior, allow for the capture of surface deformations in real-time and without the need for line-of-sight.We propose a hardware and software pipeline to fabricate flexible wearable sensors and use them to capture deformations without line of sight. Our first contribution is a low-cost fabrication pipeline to embed multiple aligned conductive layers with complex geometries into silicone compounds. Overlapping conductive areas from separate layers form local capacitors that measure dense area changes. Contrary to existing fabrication methods, the proposed technique only requires hardware that is readily available in modern fablabs. While area measurements alone are not enough to reconstruct the full 3D deformation of a surface, they become sufficient when paired with a data-driven prior. A novel semi-automatic tracking algorithm, based on an elastic surface geometry deformation, allows to capture ground-truth data with an optical mocap system, even under heavy occlusions or partially unobservable markers. The resulting dataset is used to train a regressor based on deep neural networks, directly mapping the area readings to global positions of surface vertices. We demonstrate the flexibility and accuracy of the proposed hardware and software in a series of controlled experiments, and design a prototype of wearable wrist, elbow and biceps sensors, which do not require line-of-sight and can be worn below regular clothing.

A Survey on Hand Pose Estimation with Wearable Sensors and Computer-Vision-Based Methods

“…However, the abovementioned stretch sensor gloves all have a limited number of embedded sensors (up to 15 [37]), which limits their use for full-hand pose recovery. To solve this problem, Glauser et al [40] extended the capacitive strain sensor concept of Atalay et al [38] to achieve dense area-stretch sensor arrays. Later, they designed a stretchable glove based on stretch array sensors, combined with a learned prior, to capture dense surface deformations of full hands [41] (Figure 4e).…”

Capacitivo

“…Explicit input refers to the input carried out in the fore of the user's consciousness [12]. Research on interactive fabrics has been primarily focused on explicit input using touch [1,24,33] or deformation gestures [6,28,36]. The earliest exploration of this space was the Musical Jacket [23], which allows a user to interact with a computer using a fabric-based touch keypad embroidered on a jacket.…”

“…Interactive fabrics enable numerous applications for smart everyday "things" and beyond (e.g., garments, furniture, and toys) [1,7,23,40]. However, with existing sensing techniques, input through a fabric is primarily carried out by a user performing an action, such as touching [1,23,30] or deforming [6,25,36]. This means that the fabric lacks awareness of its context of use, such as what types of objects it is in contact with.…”

Specialized CNT-based Sensor Framework for Advanced Motion Tracking

“…In such cases, stretchable sensors are deemed more appropriate, and active research is continuing in this field. Currently, stretchable sensors are built employing either capacitive [19,20,21], or resistive [22,23,24] technologies. Following this line of research, our team is developing rapid-response, widely stretchable sensors based on carbon nanotubes specialized for the detection of human motions [25,26,27].…”