Head-Mounted Augmented Reality for Explainable Robotic Wheelchair Assistance

Abstract

Robotic wheelchairs with built-in assistive features, such as shared control, are an emerging means of providing independent mobility to severely disabled individuals. However, patients often struggle to build a mental model of their wheelchair's behaviour under different environmental conditions. Motivated by the desire to help users bridge this gap in perception, we propose a novel augmented reality system using a Microsoft Hololens as a head-mounted aid for wheelchair navigation. The system displays visual feedback to the wearer as a way of explaining the underlying dynamics of the wheelchair's shared controller and its predicted future states. To investigate the influence of different interface design options, a pilot study was also conducted. We evaluated the acceptance rate and learning curve of an immersive wheelchair training regime, revealing preliminary insights into the potential beneficial and adverse nature of different augmented reality cues for assistive navigation. In particular, we demonstrate that care should be taken in the presentation of information, with effort-reducing cues for augmented information acquisition (for example, a rear-view display) being the most appreciated.

Augmented Reality Controlled Smart Wheelchair Using Dynamic Signifiers for Affordance Representation

“…While mobile AR applications have also been researched, they have only been used as a way of giving additional information to the user, for example using fiducial markers [6] and beacon technologies [7] to provide the user with a route for indoor navigation. Zolotas et al [8] have recently presented a novel AR-based interaction scenario, in which a wheelchair user can become more aware of the smart wheelchair's intentions in navigation tasks through a set of AR visualisations provided as feedback.…”

“…In addition, the built-in shared control given by this module manages the obstacle avoidance and local path planning tasks. The shared control method used by our platform is described in detail by Zolotas et al in [8].…”

“…Spatial anchors represent important points in the world that the HoloLens keeps track of over time so that they stay precisely in place [26]. These positions are periodically reported by the virtual objects inside the application and compared to their corresponding positions in the ROS map to obtain the transformation between the application coordinate system and the smart wheelchair's frame of reference as described by Zolotas et al [8].…”

“…AR User Interfaces (UIs) have the potential of introducing more natural interactions between the user and the robot that are easier to understand and learn. Even though mobile Augmented Reality (AR) interfaces for wheelchairs have been previously proposed [6], [7], [8], to the best of our knowledge these interfaces have never been used before as a way to enable control, but instead mainly as a way of providing visual aids.…”

An Augmented Reality-based Telerehabilitation Architecture for Supporting the Training of Powered Wheelchair Users

“…On the other hand, Augmented Reality (AR) can provide user interaction in a more realistic and intuitive way and safely (MANRING et al, 2020;BORRESEN et al, 2019;ABDALLAH et al, 2019;WIEDERHOLD, 2019;ECKERT;VOLMERG;FRIEDRICH, 2019;ZOLOTAS;ELSDON;DEMIRIS, 2018;MAULE et al, 2017;CAETANO et al, 2014). In doing so, the system augments the real world with digital information (e.g., pictures, videos, instructions, clinical data) enhancing the user's experience (BORRESEN et al, 2019).…”