Visual attention in egocentric field-of-view using RGB-D data

Olesova, Veronika; Benešová, Wanda; Polatsek, Patrik

doi:10.1117/12.2268617

Cited by 2 publications

(1 citation statement)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Gaze has been used successfully in the past as an interface for machines, particularly in human-computer interfaces [17], [18] and social robotics to monitor human attention, emotion and engagement [19]- [22] as well as robotic laparoscopic surgery [23]. When it comes to patients with movement disabilities, there is work on the use of gaze patterns in rehabilitation [24], for the control of 2 degrees of freedom in upper limb exoskeletons, where the patient uses gaze to direct the robot on a 2D surface.…”

Section: Introductionmentioning

confidence: 99%

Gaze-based, Context-aware Robotic System for Assisted Reaching and Grasping

Shafti

Orlov

Faisal

2019

2019 International Conference on Robotics and Automation (ICRA)

View full text Add to dashboard Cite

Assistive robotic systems endeavour to support those with movement disabilities, enabling them to move again and regain functionality. Main issue with these systems is the complexity of their low-level control, and how to translate this to simpler, higher level commands that are easy and intuitive for a human user to interact with. We have created a multimodal system, consisting of different sensing, decision making and actuating modalities, leading to intuitive, human-in-theloop assistive robotics. The system takes its cue from the user's gaze, to decode their intentions and implement low-level motion actions to achieve high-level tasks. This results in the user simply having to look at the objects of interest, for the robotic system to assist them in reaching for those objects, grasping them, and using them to interact with other objects. We present our method for 3D gaze estimation, and grammarsbased implementation of sequences of action with the robotic system. The 3D gaze estimation is evaluated with 8 subjects, showing an overall accuracy of 4.68±0.14cm. The full system is tested with 5 subjects, showing successful implementation of 100% of reach to gaze point actions and full implementation of pick and place tasks in 96%, and pick and pour tasks in 76% of cases. Finally we present a discussion on our results and what future work is needed to improve the system. Fig. 3: Geometric representation of the gaze angle and the camera frame, used for the 3D gaze point estimation.

show abstract