Electrooculography and Tactile Perception Collaborative Interface for 3D Human–Machine Interaction

Abstract

The human–machine interface (HMI) previously relied on a single perception interface that cannot realize three-dimensional (3D) interaction and convenient and accurate interaction in multiple scenes. Here, we propose a collaborative interface including electrooculography (EOG) and tactile perception for fast and accurate 3D human–machine interaction. The EOG signals are mainly used for fast, convenient, and contactless 2D (XY-axis) interaction, and the tactile sensing interface is mainly utilized for complex 2D movement control and Z-axis control in the 3D interaction. The honeycomb graphene electrodes for the EOG signal acquisition and tactile sensing array are prepared by a laser-induced process. Two pairs of ultrathin and breathable honeycomb graphene electrodes are attached around the eyes for monitoring nine different eye movements. A machine learning algorithm is designed to train and classify the nine different eye movements with an average prediction accuracy of 92.6%. Furthermore, an ultrathin (90 μm), stretchable (∼1000%), and flexible tactile sensing interface assembled by a pair of 4 × 4 planar electrode arrays is attached to the arm for 2D movement control and Z-axis interaction, which can realize single-point, multipoint and sliding touch functions. Consequently, the tactile sensing interface can achieve eight directions control and even more complex movement trajectory control. Meanwhile, the flexible and ultrathin tactile sensor exhibits an ultrahigh sensitivity of 1.428 kPa–1 in the pressure range 0–300 Pa with long-term response stability and repeatability. Therefore, the collaboration between EOG and the tactile perception interface will play an important role in rapid and accurate 3D human–machine interaction.

An Ultra-Sensitive and Multifunctional Electronic Skin with Synergetic Network of Graphene and CNT

“…Under normal conditions, the heart rate is calculated to be 72 beats min −1 , which is within the normal heart rate range of healthy people. Moreover, the SCG e-skin is attached next to the eye to monitor blinking ( Figure 4 c), indicating that the SCG e-skin can detect muscle movements caused by tiny facial expressions, which has a potential application in 3D human−machine interactions in the future [ 39 ]. In addition to the tiny signal, the SCG e-skin is used to detect large human activities.…”

Recent Progress in Flexible Pressure Sensor Arrays

“…A vital application of flexible pressure sensor arrays in HMIs is spatial pressure distribution monitoring and real-time trajectory mapping. For example, Xu et al proposed a 3D human–machine interaction by combining electrooculography (EOG) and a 4 × 4 capacitive pressure sensor array [ 110 ]. The signals of the EOG were used for convenient contactless 2D (XY-axis) interaction, and the signals of the pressure sensor array were used for complex 2D movement control and Z-axis control in the 3D interaction ( Figure 8 a).…”

Laser-Patterned Hierarchical Aligned Micro-/Nanowire Network for Highly Sensitive Multidimensional Strain Sensor

“…Wearable strain sensors monitoring electrical signals stimulated by mechanical motions have received tremendous attention in fields such as soft robotics, − motion monitoring, − and electronic skin − over the past decade. The sensing mechanisms of strain sensors include piezoelectricity, capacitance, piezoresistivity, and triboelectricity. − Piezoresistive-type strain sensors based on external strain-induced electrical resistance response have been widely used due to the merits of simple construction, facile measurement, and high sensitivity. , However, most of the existing strain sensors can merely detect uniaxial strain owing to the coupled strain responses between the parallel and perpendicular directions of applied loadings, limiting the accurate detection of complicated human movements, including direction and amplitude in real time. − …”