Machine Learning Glove Using Self‐Powered Conductive Superhydrophobic Triboelectric Textile for Gesture Recognition in VR/AR Applications

Abstract

The rapid progress of Internet of things (IoT) technology raises an imperative demand on human machine interfaces (HMIs) which provide a critical linkage between human and machines. Using a glove as an intuitive and low‐cost HMI can expediently track the motions of human fingers, resulting in a straightforward communication media of human–machine interactions. When combining several triboelectric textile sensors and proper machine learning technique, it has great potential to realize complex gesture recognition with the minimalist‐designed glove for the comprehensive control in both real and virtual space. However, humidity or sweat may negatively affect the triboelectric output as well as the textile itself. Hence, in this work, a facile carbon nanotubes/thermoplastic elastomer (CNTs/TPE) coating approach is investigated in detail to achieve superhydrophobicity of the triboelectric textile for performance improvement. With great energy harvesting and human motion sensing capabilities, the glove using the superhydrophobic textile realizes a low‐cost and self‐powered interface for gesture recognition. By leveraging machine learning technology, various gesture recognition tasks are done in real time by using gestures to achieve highly accurate virtual reality/augmented reality (VR/AR) controls including gun shooting, baseball pitching, and flower arrangement, with minimized effect from sweat during operation.

Progress inTENGtechnology—A journey from energy harvesting to nanoenergy and nanosystem

“…225 Additionally, with the aid of the machine learning technique, a TENG glove made by superhydrophobic CNT-TPE coating was reported by Wen et al as shown in Figure 7I. 226 The proposed glove not only enhanced the durability of the textile-based TENG sensor under high humidity but also demonstrated the gesture recognition capability. Moving forward, Zhu et al reported a smart glove consists of TENG based finger bending sensor and a palm sliding sensor, as well as a piezoelectric haptic stimulator ( Figure 7J).…”

“…To achieve multi-functionalities, with the aid of triboelectric materials and structural designs, advanced electrode designs can also be incorporated into conventional TENGs, resulting in applicationoriented TENG configurations, that is, operating independently in the form of real-time and situ sensing. 105,226,243,[374][375][376][377][378][379][380][381][382][383][384][385][386][387][388][389][390][391][392] Toward NENS, blue energy is one of the most important application directions, which is mainly in forms of wave energy, tidal energy, and osmotic energy harvesting. To the aspect of the wearable electronics and HMI, TENG shows the potential abilities in realizing the advanced manipulation with the digital world.…”

A Review and Perspective for the Development of Triboelectric Nanogenerator (TENG)-Based Self-Powered Neuroprosthetics

“…On the other hand, its soft nature enables the sensing of the pressing area and force. Therefore, similar soft devices are widely used for all kinds of self-powered physical sensing, such as foot stepping, hand gesture and body motion, in wearable electronics [ 72 , 73 , 74 , 75 ]. However, integration of a soft thin-film TENG device and a switch changes the whole story.…”

Triboelectric nanogenerator sensors for soft robotics aiming at digital twin applications

“…Noticeably, triboelectric nanogenerator (TENG) 27 – 30 -based sensors are more compatible with soft robotics, because the Young’s modulus of soft materials typically used in triboelectric sensors is in the same level as the silicone rubber and TPU. More importantly, self-generated sensor output in response to the strains and deformation of TENG makes the design of signal processing circuits more straightforward.…”