Human-machine interfaces (HMIs) experience increasing requirements for intuitive and effective manipulation. Current commercialized solutions of glove-based HMI are limited by either detectable motions or the huge cost on fabrication, energy, and computing power. We propose the haptic-feedback smart glove with triboelectric-based finger bending sensors, palm sliding sensor, and piezoelectric mechanical stimulators. The detection of multidirectional bending and sliding events is demonstrated in virtual space using the self-generated triboelectric signals for various degrees of freedom on human hand. We also perform haptic mechanical stimulation via piezoelectric chips to realize the augmented HMI. The smart glove achieves object recognition using machine learning technique, with an accuracy of 96%. Through the integrated demonstration of multidimensional manipulation, haptic feedback, and AI-based object recognition, our glove reveals its potential as a promising solution for low-cost and advanced human-machine interaction, which can benefit diversified areas, including entertainment, home healthcare, sports training, and medical industry.

Designing efficient sensors for soft robotics aiming at human machine interaction remains a challenge. Here, we report a smart soft-robotic gripper system based on triboelectric nanogenerator sensors to capture the continuous motion and tactile information for soft gripper. With the special distributed electrodes, the tactile sensor can perceive the contact position and area of external stimuli. The gear-based length sensor with a stretchable strip allows the continuous detection of elongation via the sequential contact of each tooth. The triboelectric sensory information collected during the operation of soft gripper is further trained by support vector machine algorithm to identify diverse objects with an accuracy of 98.1%. Demonstration of digital twin applications, which show the object identification and duplicate robotic manipulation in virtual environment according to the real-time operation of the soft-robotic gripper system, is successfully created for virtual assembly lines and unmanned warehouse applications.

The rapid progress of Internet of things (IoT) technology raises an imperative demand on human machine interfaces (HMIs) which provide a critical linkage between human and machines. Using a glove as an intuitive and low‐cost HMI can expediently track the motions of human fingers, resulting in a straightforward communication media of human–machine interactions. When combining several triboelectric textile sensors and proper machine learning technique, it has great potential to realize complex gesture recognition with the minimalist‐designed glove for the comprehensive control in both real and virtual space. However, humidity or sweat may negatively affect the triboelectric output as well as the textile itself. Hence, in this work, a facile carbon nanotubes/thermoplastic elastomer (CNTs/TPE) coating approach is investigated in detail to achieve superhydrophobicity of the triboelectric textile for performance improvement. With great energy harvesting and human motion sensing capabilities, the glove using the superhydrophobic textile realizes a low‐cost and self‐powered interface for gesture recognition. By leveraging machine learning technology, various gesture recognition tasks are done in real time by using gestures to achieve highly accurate virtual reality/augmented reality (VR/AR) controls including gun shooting, baseball pitching, and flower arrangement, with minimized effect from sweat during operation.

The past few years have witnessed the significant impacts of wearable electronics/photonics on various aspects of our daily life, for example, healthcare monitoring and treatment, ambient monitoring, soft robotics, prosthetics, flexible display, communication, human‐machine interactions, and so on. According to the development in recent years, the next‐generation wearable electronics and photonics are advancing rapidly toward the era of artificial intelligence (AI) and internet of things (IoT), to achieve a higher level of comfort, convenience, connection, and intelligence. Herein, this review provides an opportune overview of the recent progress in wearable electronics, photonics, and systems, in terms of emerging materials, transducing mechanisms, structural configurations, applications, and their further integration with other technologies. First, development of general wearable electronics and photonics is summarized for the applications of physical sensing, chemical sensing, human‐machine interaction, display, communication, and so on. Then self‐sustainable wearable electronics/photonics and systems are discussed based on system integration with energy harvesting and storage technologies. Next, technology fusion of wearable systems and AI is reviewed, showing the emergence and rapid development of intelligent/smart systems. In the last section of this review, perspectives about the future development trends of the next‐generation wearable electronics/photonics are provided, that is, toward multifunctional, self‐sustainable, and intelligent wearable systems in the AI/IoT era.

The era of artificial intelligence and internet of things is rapidly developed by recent advances in wearable electronics. Gait reveals sensory information in daily life containing personal information, regarding identification and healthcare. Current wearable electronics of gait analysis are mainly limited by high fabrication cost, operation energy consumption, or inferior analysis methods, which barely involve machine learning or implement nonoptimal models that require massive datasets for training. Herein, we developed low-cost triboelectric intelligent socks for harvesting waste energy from low-frequency body motions to transmit wireless sensory data. The sock equipped with self-powered functionality also can be used as wearable sensors to deliver information, regarding the identity, health status, and activity of the users. To further address the issue of ineffective analysis methods, an optimized deep learning model with an end-to-end structure on the socks signals for the gait analysis is proposed, which produces a 93.54% identification accuracy of 13 participants and detects five different human activities with 96.67% accuracy. Toward practical application, we map the physical signals collected through the socks in the virtual space to establish a digital human system for sports monitoring, healthcare, identification, and future smart home applications.

Toward smart building and smart home, floor as one of our most frequently interactive interfaces can be implemented with embedded sensors to extract abundant sensory information without the video-taken concerns. Yet the previously developed floor sensors are normally of small scale, high implementation cost, large power consumption, and complicated device configuration. Here we show a smart floor monitoring system through the integration of self-powered triboelectric floor mats and deep learning-based data analytics. The floor mats are fabricated with unique “identity” electrode patterns using a low-cost and highly scalable screen printing technique, enabling a parallel connection to reduce the system complexity and the deep-learning computational cost. The stepping position, activity status, and identity information can be determined according to the instant sensory data analytics. This developed smart floor technology can establish the foundation using floor as the functional interface for diverse applications in smart building/home, e.g., intelligent automation, healthcare, and security.

Advancements of virtual reality technology pave the way for developing wearable devices to enable somatosensory sensation, which can bring more comprehensive perception and feedback in the metaverse-based virtual society. Here, we propose augmented tactile-perception and haptic-feedback rings with multimodal sensing and feedback capabilities. This highly integrated ring consists of triboelectric and pyroelectric sensors for tactile and temperature perception, and vibrators and nichrome heaters for vibro- and thermo-haptic feedback. All these components integrated on the ring can be directly driven by a custom wireless platform of low power consumption for wearable/portable scenarios. With voltage integration processing, high-resolution continuous finger motion tracking is achieved via the triboelectric tactile sensor, which also contributes to superior performance in gesture/object recognition with artificial intelligence analysis. By fusing the multimodal sensing and feedback functions, an interactive metaverse platform with cross-space perception capability is successfully achieved, giving people a face-to-face like immersive virtual social experience.

With the fast development of the fifth-generation cellular network technology (5G), the future sensors and microelectromechanical systems (MEMS)/nanoelectromechanical systems (NEMS) are presenting a more and more critical role to provide information in our daily life. This review paper introduces the development trends and perspectives of the future sensors and MEMS/NEMS. Starting from the issues of the MEMS fabrication, we introduced typical MEMS sensors for their applications in the Internet of Things (IoTs), such as MEMS physical sensor, MEMS acoustic sensor, and MEMS gas sensor. Toward the trends in intelligence and less power consumption, MEMS components including MEMS/NEMS switch, piezoelectric micromachined ultrasonic transducer (PMUT), and MEMS energy harvesting were investigated to assist the future sensors, such as event-based or almost zero-power. Furthermore, MEMS rigid substrate toward NEMS flexible-based for flexibility and interface was discussed as another important development trend for next-generation wearable or multi-functional sensors. Around the issues about the big data and human-machine realization for human beings’ manipulation, artificial intelligence (AI) and virtual reality (VR) technologies were finally realized using sensor nodes and its wave identification as future trends for various scenarios.