Human–Machine Interface: Multiclass Classification by Machine Learning on 1D EOG Signals for the Control of an Omnidirectional Robot

Pérez-Reynoso, Francisco; Rodríguez-Guerrero, Liliam; Salgado-Ramírez, Julio César; Ortega-Palacios, R.

doi:10.3390/s21175882

Cited by 10 publications

(4 citation statements)

References 51 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Conventional electrodes, wet or dry silver/silver chloride (Ag/AgCl), are generally used to measure EOG signals [ 76 , 77 , 78 , 79 , 80 ]. For example, wet Ag/AgCl electrodes are used for the analysis of various activity recognition fields [ 65 , 81 , 82 ] or HMI controllers [ 37 , 60 , 62 , 83 ]. Dry flat Ag/AgCl electrodes are used on various wearable platforms such as eyeglasses [ 35 ], head caps [ 55 ], and goggles [ 34 ].…”

Section: Eog Signalsmentioning

confidence: 99%

Section: Eog Signalsmentioning

confidence: 99%

“…Recent advances in microfabrication and print technologies enabled new ways to design micro-patterned electrodes (gold [ 1 , 30 ] and graphene [ 11 ]). Due to the development of these technologies and 3D printing, designing wearable platforms has become possible, such as eyeglass types [ 5 , 31 , 32 , 33 , 34 , 35 , 36 , 37 , 38 , 39 ], face mask types [ 7 , 40 , 41 , 42 ], ear plug types [ 43 , 44 , 45 , 46 , 47 ], and headband types [ 48 , 49 , 50 , 51 , 52 , 53 , 54 , 55 , 56 , 57 , 58 ] for various applications. Previously, various controllers for an HMI such as wheelchairs [ 1 , 4 , 51 , 52 ], drones [ 11 , 59 ], game interfaces [ 5 , 36 , 47 , 60 , 61 ], and virtual keyboards [ 34 , 38 , 51 , …”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Advances in Materials, Sensors, and Integrated Systems for Monitoring Eye Movements

et al. 2022

View full text Add to dashboard Cite

Eye movements show primary responses that reflect humans' voluntary intention and conscious selection. Because visual perception is one of the fundamental sensory interactions in the brain, eye movements contain critical information regarding physical/psychological health, perception, intention, and preference. With the advancement of wearable device technologies, the performance of monitoring eye tracking has been significantly improved. It also has led to myriad applications for assisting and augmenting human activities. Among them, electrooculograms, measured by skin-mounted electrodes, have been widely used to track eye motions accurately. In addition, eye trackers that detect reflected optical signals offer alternative ways without using wearable sensors. This paper outlines a systematic summary of the latest research on various materials, sensors, and integrated systems for monitoring eye movements and enabling human-machine interfaces. Specifically, we summarize recent developments in soft materials, biocompatible materials, manufacturing methods, sensor functions, systems' performances, and their applications in eye tracking. Finally, we discuss the remaining challenges and suggest research directions for future studies.

show abstract

Section: Eog Signalsmentioning

confidence: 99%

Section: Eog Signalsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Advances in Materials, Sensors, and Integrated Systems for Monitoring Eye Movements

et al. 2022

View full text Add to dashboard Cite

show abstract

“…Kang et al applied an ensemble network with attention mechanisms [39]. Pérez-Reynoso et al developed an EOG-controlled robotic system [40]. However, gap still exists in how deep learning can further decode complex patterns and how real-time inference can be deployed.…”

Section: Introductionmentioning

confidence: 99%

eyeSay: Brain Visual Dynamics Decoding With Deep Learning & Edge Computing

Zou

Zhang

2022

IEEE Trans. Neural Syst. Rehabil. Eng.

View full text Add to dashboard Cite

Brain visual dynamics encode rich functional and biological patterns of the neural system, and if decoded, are of great promise for many applications such as intention understanding, cognitive load quantization and neural disorder measurement. We here focus on the understanding of the brain visual dynamics for the Amyotrophic lateral sclerosis (ALS) population, and propose a novel system that allows these so-called 'lock-in' patients to 'speak' with their brain visual movements. More specifically, we propose an intelligent system to decode the eye bio-potential signal, Electrooculogram (EOG), thereby understanding the patients' intention. We first propose to leverage a deep learning framework for automatic feature learning and classification of the brain visual dynamics, aiming to translate the EOG to meaningful words. We afterwards design and develop an edge computing platform on the smart phone, which can execute the deep learning algorithm, visualize the brain visual dynamics, and demonstrate the edge inference results, all in real-time. Evaluated on 4,500 trials of brain visual movements performed by multiple users, our novel system has demonstrated a high eyeword recognition rate up to 90.47%. The system is demonstrated to be intelligent, effective and convenient for decoding brain visual dynamics for ALS patients. This research thus is expected to greatly advance the decoding and understanding of brain visual dynamics, by leveraging machine learning and edge computing innovations.

show abstract

Persistent Human–Machine Interfaces for Robotic Arm Control Via Gaze and Eye Direction Tracking

Ban

Lee

Chang

et al. 2023

Advanced Intelligent Systems

View full text Add to dashboard Cite

Recent advances in sensors and electronics have enabled electrooculogram (EOG) detection systems for capturing eye movements. However, EOG signals are susceptible to the sensor's skin‐contact quality, limiting the precise detection of eye angles and gaze. Herein, a two‐camera eye‐tracking system and a data classification method for persistent human–machine interfaces (HMIs) are introduced. Machine‐learning technology is used for a continuous real‐time classification of gaze and eye directions, to precisely control a robotic arm. In addition, a deep‐learning algorithm for classifying eye directions is developed and the pupil center‐corneal reflection method of an eye tracker for gaze tracking is utilized. A supervisory control and data acquisition architecture that can be universally applied to any screen‐based HMI task are used by the system. It is shown in the study that the classification algorithm using deep learning enables exceptional accuracy (99.99%) with the number of actions per command (≥64), the highest performance compared to other HMI systems. Demonstrating real‐time control of a robotic arm captures the unique advantages of the precise eye‐tracking system for playing chess and manipulating dice. Overall, this paper shows the HMI system's potential for remote control of surgery robots, warehouse systems, and construction tools.

show abstract

Human–Machine Interface: Multiclass Classification by Machine Learning on 1D EOG Signals for the Control of an Omnidirectional Robot

Cited by 10 publications

References 51 publications

Advances in Materials, Sensors, and Integrated Systems for Monitoring Eye Movements

Advances in Materials, Sensors, and Integrated Systems for Monitoring Eye Movements

eyeSay: Brain Visual Dynamics Decoding With Deep Learning & Edge Computing

Persistent Human–Machine Interfaces for Robotic Arm Control Via Gaze and Eye Direction Tracking

Contact Info

Product

Resources

About