Face-Computer Interface (FCI): Intent Recognition Based on Facial Electromyography (fEMG) and Online Human-Computer Interface With Audiovisual Feedback

Abstract

Patients who have lost limb control ability, such as upper limb amputation and high paraplegia, are usually unable to take care of themselves. Establishing a natural, stable, and comfortable human-computer interface (HCI) for controlling rehabilitation assistance robots and other controllable equipments will solve a lot of their troubles. In this study, a complete limbs-free face-computer interface (FCI) framework based on facial electromyography (fEMG) including offline analysis and online control of mechanical equipments was proposed. Six facial movements related to eyebrows, eyes, and mouth were used in this FCI. In the offline stage, 12 models, eight types of features, and three different feature combination methods for model inputing were studied and compared in detail. In the online stage, four well-designed sessions were introduced to control a robotic arm to complete drinking water task in three ways (by touch screen, by fEMG with and without audio feedback) for verification and performance comparison of proposed FCI framework. Three features and one model with an average offline recognition accuracy of 95.3%, a maximum of 98.8%, and a minimum of 91.4% were selected for use in online scenarios. In contrast, the way with audio feedback performed better than that without audio feedback. All subjects completed the drinking task in a few minutes with FCI. The average and smallest time difference between touch screen and fEMG under audio feedback were only 1.24 and 0.37 min, respectively.

SeNic: An Open Source Dataset for sEMG-Based Gesture Recognition in Non-Ideal Conditions

“…According to our previous experience of sEMG and its application research [5,[44][45][46][47][48], especially for research under non-ideal conditions [11,13,49], designing experimental and collecting experimental data will consume a lot of time and energy. However, what is more important in the research process is usually the analysis of sEMG features and the improvement of models.…”

Using principles of motor control to analyze performance of human machine interfaces

“…In recent years, there have been significant advances in the control of biomechatronic devices driven by biological signals derived from the user 1 – 9 . Various biological signals have been used for gesture intent recognition to drive such biomechatronic devices, including electrical signals such as surface electromyography (sEMG) 10 – 12 , electroencephalography 13 , 14 , electrocorticography 15 , 16 , as well as mechanical signals such as mechanomyography 17 and sonomyography 18 – 24 . These signal extraction techniques have found broad applications in rehabilitation engineering 25 – 27 .…”

Using Principles of Motor Control to Analyze Performance of Human Machine Interfaces

“…In recent years, there have been significant advances in the control of biomechatronic devices driven by biological signals derived from the user [1][2][3][4][5][6][7][8][9] . Various biological signals have been used for gesture intent recognition to drive such biomechatronic devices, including electrical signals such as surface electromyography [10][11][12] , electroencephalography 13,14 , electrocorticography 15,16 , as well as mechanical signals such as mechanomyography 17 and sonomyography [18][19][20][21][22][23][24] . These signal extraction techniques have found broad applications in rehabilitation engineering [25][26][27] .…”