Eye-Tracking Analysis with Deep Learning Method

Firat, Taner Tuncer

doi:10.1109/3ict53449.2021.9581943

Cited by 4 publications

(2 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Brain-computer interfaces (BCI) have revolutionized our interactions with technology, creating direct links between the brain and electronic devices [ 18 ]. Incorporating various noninvasive methods such as EEG [ 19 , 20 ], fNIRS [ 21 ], eye-tracking [ 22 , 23 ], and VR/AR integrations [ 24 , 25 ], BCIs promise wide-ranging applications. These include facilitating communication for those with disabilities [ 19 , 26 ] and enriching immersive gaming and virtual reality experiences [ 27 ], as well as roles in disease diagnosis [ 28 , 29 ] and mental state monitoring [ 30 , 31 ].…”

Section: Related Workmentioning

confidence: 99%

EF-Net: Mental State Recognition by Analyzing Multimodal EEG-fNIRS via CNN

Arif,

Wang,

Yin

et al. 2024

Sensors

View full text Add to dashboard Cite

Analysis of brain signals is essential to the study of mental states and various neurological conditions. The two most prevalent noninvasive signals for measuring brain activities are electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS). EEG, characterized by its higher sampling frequency, captures more temporal features, while fNIRS, with a greater number of channels, provides richer spatial information. Although a few previous studies have explored the use of multimodal deep-learning models to analyze brain activity for both EEG and fNIRS, subject-independent training–testing split analysis remains underexplored. The results of the subject-independent setting directly show the model’s ability on unseen subjects, which is crucial for real-world applications. In this paper, we introduce EF-Net, a new CNN-based multimodal deep-learning model. We evaluate EF-Net on an EEG-fNIRS word generation (WG) dataset on the mental state recognition task, primarily focusing on the subject-independent setting. For completeness, we report results in the subject-dependent and subject-semidependent settings as well. We compare our model with five baseline approaches, including three traditional machine learning methods and two deep learning methods. EF-Net demonstrates superior performance in both accuracy and F1 score, surpassing these baselines. Our model achieves F1 scores of 99.36%, 98.31%, and 65.05% in the subject-dependent, subject-semidependent, and subject-independent settings, respectively, surpassing the best baseline F1 scores by 1.83%, 4.34%, and 2.13% These results highlight EF-Net’s capability to effectively learn and interpret mental states and brain activity across different and unseen subjects.

show abstract

Section: Related Workmentioning

confidence: 99%

EF-Net: Mental State Recognition by Analyzing Multimodal EEG-fNIRS via CNN

Arif,

Wang,

Yin

et al. 2024

Sensors

View full text Add to dashboard Cite

show abstract

“…Dliber et al used the AlexNet network structure for eye-movement direction determination and trained the network model using a private dataset. The visualization results showed that the recognition accuracy of the model reached 97.88% [26]. Moayad Mokatren et al designed a faster region-based convolutional neural network (RCNN) to detect pupils in infrared and RGB images.…”

Section: Introductionmentioning

confidence: 99%

Eye-Gaze Controlled Wheelchair Based on Deep Learning

Huang

Liu

et al. 2023

Sensors

View full text Add to dashboard Cite

In this paper, we design a technologically intelligent wheelchair with eye-movement control for patients with ALS in a natural environment. The system consists of an electric wheelchair, a vision system, a two-dimensional robotic arm, and a main control system. The smart wheelchair obtains the eye image of the controller through a monocular camera and uses deep learning and an attention mechanism to calculate the eye-movement direction. In addition, starting from the relationship between the trajectory of the joystick and the wheelchair speed, we establish a motion acceleration model of the smart wheelchair, which reduces the sudden acceleration of the smart wheelchair during rapid motion and improves the smoothness of the motion of the smart wheelchair. The lightweight eye-movement recognition model is transplanted into an embedded AI controller. The test results show that the accuracy of eye-movement direction recognition is 98.49%, the wheelchair movement speed is up to 1 m/s, and the movement trajectory is smooth, without sudden changes.

show abstract