Encoding of multi-modal emotional information via personalized skin-integrated wireless facial interface

Abstract

Human affects such as emotions, moods, feelings are increasingly being considered as key parameter to enhance the interaction of human with diverse machines and systems. However, their intrinsically abstract and ambiguous nature make it challenging to accurately extract and exploit the emotional information. Here, we develop a multi-modal human emotion recognition system which can efficiently utilize comprehensive emotional information by combining verbal and non-verbal expression data. This system is composed of personalized skin-integrated facial interface (PSiFI) system that is self-powered, facile, stretchable, transparent, featuring a first bidirectional triboelectric strain and vibration sensor enabling us to sense and combine the verbal and non-verbal expression data for the first time. It is fully integrated with a data processing circuit for wireless data transfer allowing real-time emotion recognition to be performed. With the help of machine learning, various human emotion recognition tasks are done accurately in real time even while wearing mask and demonstrated digital concierge application in VR environment.

Flexible Self-Powered Low-Decibel Voice Recognition Mask

“…The following supporting information can be downloaded at: https://www.mdpi.com/article/10.3390/s24103007/s1, Figure S1 S1: Progress of TENG in Speech Recognition Applications; Table S2:The comparison of current speech recognition technologies; Table S3:Experimental instruments and equipment. The references [39][40][41]55,[64][65][66][67][68][69][70][71][72][73][74][75][76][77] in the main text are cited in the Supplementary Information.…”

Advances in magnetic-assisted triboelectric nanogenerators: structures, materials and self-sensing systems

“…TENGs function as mechanical transducers, leveraging the interplay between triboelectric and electrostatic induction effects. In this context, the electrical output of a device is chiefly influenced by various external mechanical stimuli, encompassing displacement [158][159][160], speed [161][162][163], force [164][165][166], impulse [167,168], frequency [169,170], and environmental conditions [171,172]. This inherent sensitivity to mechanical stimulation, along with its well-defined parameter characteristics, makes the TENG an ideal candidate for deployment as a self-powered mechanical sensor.…”

Multiple physical crosslinked highly adhesive and conductive hydrogels for human motion and electrophysiological signal monitoring

“…This impedance leads to further signal loss and distortion, significantly impacting the acquisition of cutaneous electrophysiological signals. 16–20…”