Inference of user-intention in remote robot wheelchair assistance using multimodal interfaces

Schettino, Vinicius; Demiris, Yiannis

doi:10.1109/iros40897.2019.8968203

Cited by 6 publications

(4 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, data were collected using natural eye-hand coordination, with participants executing the task themselves, which represents a different situation from a teleoperation scenario both on a perceptual and action control level. In the context of assistive robotics, a number of other studies have also considered gaze information (at times combined with multimodal interfaces, such as BCI and haptic feedback) to operate robotic limbs and wheelchairs (Schettino and Demiris, 2019;Zeng et al, 2020). Often in these cases, the gaze is used to implicitly but actively point the system to the object the impaired user wants the robot to interact with Wang et al, 2018;Shafti et al, 2019).…”

Section: Related Workmentioning

confidence: 99%

Gaze-Based Intention Estimation for Shared Autonomy in Pick-and-Place Tasks

Fuchs

Belardinelli

2021

Front. Neurorobot.

View full text Add to dashboard Cite

Shared autonomy aims at combining robotic and human control in the execution of remote, teleoperated tasks. This cooperative interaction cannot be brought about without the robot first recognizing the current human intention in a fast and reliable way so that a suitable assisting plan can be quickly instantiated and executed. Eye movements have long been known to be highly predictive of the cognitive agenda unfolding during manual tasks and constitute, hence, the earliest and most reliable behavioral cues for intention estimation. In this study, we present an experiment aimed at analyzing human behavior in simple teleoperated pick-and-place tasks in a simulated scenario and at devising a suitable model for early estimation of the current proximal intention. We show that scan paths are, as expected, heavily shaped by the current intention and that two types of Gaussian Hidden Markov Models, one more scene-specific and one more action-specific, achieve a very good prediction performance, while also generalizing to new users and spatial arrangements. We finally discuss how behavioral and model results suggest that eye movements reflect to some extent the invariance and generality of higher-level planning across object configurations, which can be leveraged by cooperative robotic systems.

show abstract

Section: Related Workmentioning

confidence: 99%

Gaze-Based Intention Estimation for Shared Autonomy in Pick-and-Place Tasks

Fuchs

Belardinelli

2021

Front. Neurorobot.

View full text Add to dashboard Cite

show abstract

“…Manipulators that recognize the intention of the user’s movement are also presented to make controlling the wheelchair easier [ 43 ]. Even control devices that implement haptic [ 44 ] and visual [ 45 ] feedback are given. Unfortunately, this technology is not designed to rehabilitate this kind of population but is limited to assisting.…”

Section: Related Workmentioning

confidence: 99%

Semi-Remote Gait Assistance Interface: A Joystick with Visual Feedback Capabilities for Therapists

Gomez-Vargas

et al. 2021

Sensors

View full text Add to dashboard Cite

The constant growth of pathologies affecting human mobility has led to developing of different assistive devices to provide physical and cognitive assistance. Smart walkers are a particular type of these devices since they integrate navigation systems, path-following algorithms, and user interaction modules to ensure natural and intuitive interaction. Although these functionalities are often implemented in rehabilitation scenarios, there is a need to actively involve the healthcare professionals in the interaction loop while guaranteeing safety for them and patients. This work presents the validation of two visual feedback strategies for the teleoperation of a simulated robotic walker during an assisted navigation task. For this purpose, a group of 14 clinicians from the rehabilitation area formed the validation group. A simple path-following task was proposed, and the feedback strategies were assessed through the kinematic estimation error (KTE) and a usability survey. A KTE of 0.28 m was obtained for the feedback strategy on the joystick. Additionally, significant differences were found through a Mann–Whitney–Wilcoxon test for the perception of behavior and confidence towards the joystick according to the modes of interaction (p-values of 0.04 and 0.01, respectively). The use of visual feedback with this tool contributes to research areas such as remote management of therapies and monitoring rehabilitation of people’s mobility.

show abstract

“…[5] shows how touch and tangible interfaces could be used to facilitate the remote control of robots in hazardous situations. And [6] showed how multimodal interfaces based on gaze estimation, SLAM, and haptic feedback can be integrated to help a remote assistant infer the intention of a wheelchair driver.…”

Section: Introductionmentioning

confidence: 99%

Combining haptics and inertial motion capture to enhance remote control of a dual-arm robot

Girbés-Juan

Schettino

Gracia

et al. 2022

J Multimodal User Interfaces

Self Cite

View full text Add to dashboard Cite

High dexterity is required in tasks in which there is contact between objects, such as surface conditioning (wiping, polishing, scuffing, sanding, etc.), specially when the location of the objects involved is unknown or highly inaccurate because they are moving, like a car body in automotive industry lines. These applications require the human adaptability and the robot accuracy. However, sharing the same workspace is not possible in most cases due to safety issues. Hence, a multi-modal teleoperation system combining haptics and an inertial motion capture system is introduced in this work. The human operator gets the sense of touch thanks to haptic feedback, whereas using the motion capture device allows more naturalistic movements. Visual feedback assistance is also introduced to enhance immersion. A Baxter dual-arm robot is used to offer more flexibility and manoeuvrability, allowing to perform two independent operations simultaneously. Several tests have been carried out to assess the proposed system. As it is shown by the experimental results, the task duration is reduced and the overall performance improves thanks to the proposed teleoperation method.

show abstract

Inference of user-intention in remote robot wheelchair assistance using multimodal interfaces

Cited by 6 publications

References 16 publications

Gaze-Based Intention Estimation for Shared Autonomy in Pick-and-Place Tasks

Gaze-Based Intention Estimation for Shared Autonomy in Pick-and-Place Tasks

Semi-Remote Gait Assistance Interface: A Joystick with Visual Feedback Capabilities for Therapists

Combining haptics and inertial motion capture to enhance remote control of a dual-arm robot

Contact Info

Product

Resources

About