Real-Time Eye-Gaze Based Interaction for Human Intention Prediction and Emotion Analysis

He, Hao; She, Yingying; Xiahou, Jianbing; Yao, Junfeng; Li, Jun; Hong, Qingqi; Ji, Yingxuan

doi:10.1145/3208159.3208180

Cited by 15 publications

(3 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…They can provide information about user behavior and states. Eye gaze is a functional interaction to generate patterns for identifying the cognitive information of the users (He et al, 2018).…”

Section: Related Workmentioning

confidence: 99%

Using Gaze-based Interaction to Alleviate Situational Mobility Impairment in Extended Reality

Ghasemi

Jeong

2022

Proceedings of the Human Factors and Ergonomics Society Annual

View full text Add to dashboard Cite

Significant advancements of eye-tracking technology in extended reality (XR) head-mounted displays have increased the interest in gaze-based interactions. The benefits of gaze interaction proved that it could be a suitable alternative for hand-based interactions when users face situations where they must maintain their position due to mobility impairment. This study aims to assess the user experience of the gaze-based interaction, compared to hand-based interaction, in two movement conditions of static and dynamic. Twenty-four participants took part in this study, and their experience was evaluated in terms of perceived workload, usability, and performance. The results show that gaze-based interactions significantly outperform the hand-based interaction in terms of perceived workload and usability in case of limited mobility. Also, the user performance is significantly higher in gaze-based modes under situational impairment. The findings of this study can be used for designing XR interfaces considering the situation in which the task is performed.

show abstract

Section: Related Workmentioning

confidence: 99%

Using Gaze-based Interaction to Alleviate Situational Mobility Impairment in Extended Reality

Ghasemi

Jeong

2022

Proceedings of the Human Factors and Ergonomics Society Annual

View full text Add to dashboard Cite

show abstract

“…Gaze has been used successfully in the past as an interface for machines, particularly in human-computer interfaces [17], [18] and social robotics to monitor human attention, emotion and engagement [19]- [22] as well as robotic laparoscopic surgery [23]. When it comes to patients with movement disabilities, there is work on the use of gaze patterns in rehabilitation [24], for the control of 2 degrees of freedom in upper limb exoskeletons, where the patient uses gaze to direct the robot on a 2D surface.…”

Section: Introductionmentioning

confidence: 99%

Gaze-based, Context-aware Robotic System for Assisted Reaching and Grasping

Shafti

Orlov

Faisal

2019

2019 International Conference on Robotics and Automation (ICRA)

View full text Add to dashboard Cite

Assistive robotic systems endeavour to support those with movement disabilities, enabling them to move again and regain functionality. Main issue with these systems is the complexity of their low-level control, and how to translate this to simpler, higher level commands that are easy and intuitive for a human user to interact with. We have created a multimodal system, consisting of different sensing, decision making and actuating modalities, leading to intuitive, human-in-theloop assistive robotics. The system takes its cue from the user's gaze, to decode their intentions and implement low-level motion actions to achieve high-level tasks. This results in the user simply having to look at the objects of interest, for the robotic system to assist them in reaching for those objects, grasping them, and using them to interact with other objects. We present our method for 3D gaze estimation, and grammarsbased implementation of sequences of action with the robotic system. The 3D gaze estimation is evaluated with 8 subjects, showing an overall accuracy of 4.68±0.14cm. The full system is tested with 5 subjects, showing successful implementation of 100% of reach to gaze point actions and full implementation of pick and place tasks in 96%, and pick and pour tasks in 76% of cases. Finally we present a discussion on our results and what future work is needed to improve the system. Fig. 3: Geometric representation of the gaze angle and the camera frame, used for the 3D gaze point estimation.

show abstract

“…Eye gaze based systems have been examined for interface control [1] and evaluation [2,3] since the 1980s, shortly after computer technology advanced to the point of making the creation of such systems possible. Gaze-based interaction has many applications in a range of fields such as engineering and human-computer interaction and offers many opportunities as well as challenges for research [4,5]. Users will generally look at what they wish to interact with, and even reliably do so before the target is reached using conventional mouse movement [6].…”

Section: Introductionmentioning

confidence: 99%

Symmetric Evaluation of Multimodal Human–Robot Interaction with Gaze and Standard Control

et al. 2018

View full text Add to dashboard Cite

Control of robot arms is often required in engineering and can be performed by using different methods. This study examined and symmetrically compared the use of a controller, eye gaze tracker and a combination thereof in a multimodal setup for control of a robot arm. Tasks of different complexities were defined and twenty participants completed an experiment using these interaction modalities to solve the tasks. More specifically, there were three tasks: the first was to navigate a chess piece from a square to another pre-specified square; the second was the same as the first task, but required more moves to complete; and the third task was to move multiple pieces to reach a solution to a pre-defined arrangement of the pieces. Further, while gaze control has the potential to be more intuitive than a hand controller, it suffers from limitations with regard to spatial accuracy and target selection. The multimodal setup aimed to mitigate the weaknesses of the eye gaze tracker, creating a superior system without simply relying on the controller. The experiment shows that the multimodal setup improves performance over the eye gaze tracker alone ( p < 0.05 ) and was competitive with the controller only setup, although did not outperform it ( p > 0.05 ).

show abstract

Real-Time Eye-Gaze Based Interaction for Human Intention Prediction and Emotion Analysis

Cited by 15 publications

References 19 publications

Using Gaze-based Interaction to Alleviate Situational Mobility Impairment in Extended Reality

Using Gaze-based Interaction to Alleviate Situational Mobility Impairment in Extended Reality

Gaze-based, Context-aware Robotic System for Assisted Reaching and Grasping

Symmetric Evaluation of Multimodal Human–Robot Interaction with Gaze and Standard Control

Contact Info

Product

Resources

About