Symmetric Evaluation of Multimodal Human–Robot Interaction with Gaze and Standard Control

Abstract

Control of robot arms is often required in engineering and can be performed by using different methods. This study examined and symmetrically compared the use of a controller, eye gaze tracker and a combination thereof in a multimodal setup for control of a robot arm. Tasks of different complexities were defined and twenty participants completed an experiment using these interaction modalities to solve the tasks. More specifically, there were three tasks: the first was to navigate a chess piece from a square to another pre-specified square; the second was the same as the first task, but required more moves to complete; and the third task was to move multiple pieces to reach a solution to a pre-defined arrangement of the pieces. Further, while gaze control has the potential to be more intuitive than a hand controller, it suffers from limitations with regard to spatial accuracy and target selection. The multimodal setup aimed to mitigate the weaknesses of the eye gaze tracker, creating a superior system without simply relying on the controller. The experiment shows that the multimodal setup improves performance over the eye gaze tracker alone ( p < 0.05 ) and was competitive with the controller only setup, although did not outperform it ( p > 0.05 ).

A scoping review of gaze and eye tracking-based control methods for assistive robotic arms

“…In the experiment by Jones et al (2018) , the robot was used to play chess at different levels of difficulty. The two input modalities, eye tracking and joystick, were compared.…”

“…In this AR environment, colored rectangles were presented as visual feedback to help sort colored objects. Others were visualizations of virtual button presses ( Kim et al, 2001 ), tactile feedback in a multimodal shoulder control ( Bien et al, 2004 ), and highlighting as well as text descriptions ( Iáñez et al, 2010 ; McMullen et al, 2014 ; Jones et al, 2018 ), as shown in Figure 3 .…”

“…The resulting risk of the Midas Touch Problem can be reduced by a well choosen dwell time of about 200–700 ms. The Midas Touch Problem is reported in several included publications ( Bien et al, 2004 ; Drewes and Schmidt, 2007 ; McMullen et al, 2014 ; Velichkovsky et al, 2014 ; Alsharif et al, 2016 ; Dziemian et al, 2016 ; Webb et al, 2016 ; Meena et al, 2017 ; Jones et al, 2018 ; Stalljann et al, 2020 ). In some experiments, dwell time leads either to pleasant or irritating situations.…”

A Visual Feedback Supported Intelligent Assistive Technique for Amyotrophic Lateral Sclerosis Patients

“…[9][10][11][12] Fruitful results of eye-tracking-based CAI systems for ALS patients have been obtained. [13][14][15][16][17][18] For example, scientists at Technical University of Denmark presented a robotic teleoperation system based on gaze interaction, in which users can utilize gaze to remotely control the telerobot for achieving the path planning of maze map. [17] An eye-control-based wheelchair system is developed by researchers from the University of New York, in which eye movement is interpreted as control information that used for performing functions such as navigation and object recognition.…”

A User Study of a Gaze Window User Interface

“…To do this, the authors used the dwell time interaction which distinguishes the interaction gaze from observing gaze by the time the user gazes on an object or area. Jones et al [6] conducted a study that compared the performance eye gaze tracker with that of two other methods in control of robot arms. Some other researchers used gaze for controlling the view when looking at the edges of the screen or virtual arrow buttons [10,11].…”