Highly Robust and Wearable Facial Expression Recognition via Deep-Learning-Assisted, Soft Epidermal Electronics

Abstract

The facial expressions are a mirror of the elusive emotion hidden in the mind, and thus, capturing expressions is a crucial way of merging the inward world and virtual world. However, typical facial expression recognition (FER) systems are restricted by environments where faces must be clearly seen for computer vision, or rigid devices that are not suitable for the time-dynamic, curvilinear faces. Here, we present a robust, highly wearable FER system that is based on deep-learning-assisted, soft epidermal electronics. The epidermal electronics that can fully conform on faces enable high-fidelity biosignal acquisition without hindering spontaneous facial expressions, releasing the constraint of movement, space, and light. The deep learning method can significantly enhance the recognition accuracy of facial expression types and intensities based on a small sample. The proposed wearable FER system is superior for wide applicability and high accuracy. The FER system is suitable for the individual and shows essential robustness to different light, occlusion, and various face poses. It is totally different from but complementary to the computer vision technology that is merely suitable for simultaneous FER of multiple individuals in a specific place. This wearable FER system is successfully applied to human-avatar emotion interaction and verbal communication disambiguation in a real-life environment, enabling promising human-computer interaction applications.

E-Tattoos: Toward Functional but Imperceptible Interfacing with Human Skin

“…Utilizing an adhesive medical epidermal patch, such as Tegaderm, can enhance the lifespan of temporary ultrathin tattoo electrodes on the skin by providing additional adhesion force and protection. Tegaderm is a polyurethane film-based transparent adhesive medical patch with exceptional breathability (WVTR ∼14.47 ± 0.30 g·m 2 ·h), thinness (∼47 μm), strong adhesion (∼5.93 N/cm), and notable mechanical properties (modulus ∼7 MPa). , Tegaderm can serve as either substrate for the electronics or surface encapsulation to protect preskin-laminated e-tattoos (Figure c). − Despite enabling robust lamination and extending the endurance of e-tattoos on the skin, Tegaderm is nonreusable, limiting the multiple uses of the e-tattoos.…”

Flexible electronics for cardiovascular healthcare monitoring

“…For example, Zhuang et al. 136 introduced a very interesting paradigm that realized facial expression recognition via deep-learning-assisted epidermal electrodes.…”

Response Regulation for Epidermal Fabric Strain Sensors via Mechanical Strategy

“…Recent advances in flexible electronics have paved the way for a wide variety of important applications, [ 1–4 ] such as low‐power radio communication systems, [ 5 ] efficient energy harvesting/storage devices, [ 6,7 ] human‐machine interaction technologies, [ 8–11 ] miniaturized electronic/optoelectronic components, 12–14] electronic microfliers [ 15 ] and smart electronic skin for robots [ 16 ] and aircraft. [ 17,18 ] At the same time, low‐cost materials and manufacturing methods, [ 19–24 ] fabrication methods to prepare deformable sensors or functional circuits on 3D freeform surfaces, [ 25–29 ] and self‐powered (standalone) stretchable sensing platforms have also been flourishing.…”