Benchmarking Wearable Robots: Challenges and Recommendations From Functional, User Experience, and Methodological Perspectives

Abstract

Wearable robots (WRs) are increasingly moving out of the labs toward real-world applications. In order for WRs to be effectively and widely adopted by end-users, a common benchmarking framework needs to be established. In this article, we outline the perspectives that in our opinion are the main determinants of this endeavor, and exemplify the complex landscape into three areas. The first perspective is related to quantifying the technical performance of the device and the physical impact of the device on the user. The second one refers to the understanding of the user's perceptual, emotional, and cognitive experience of (and with) the technology. The third one proposes a strategic path for a global benchmarking methodology, composed by reproducible experimental procedures representing real-life conditions. We hope that this paper can enable developers, researchers, clinicians and end-users to efficiently identify the most promising directions for validating their technology and drive future research efforts in the short and medium term.

An Analysis of Usability Evaluation Practices And Contexts of Use In Wearable Robotics

“…Nevertheless, our survey data allowed a novel understanding and comprehensive review of WRD applications, their development stages and current usability evaluation practices. We could further elucidate the need for evaluation benchmarks and development guidelines, as it has been detailed and called for in recent works [19,25]. Our analysis may even support the benchmarking endeavors of initiatives such as the EUROBENCH project [41], the Exo Technology Center of Excellence from ASTM International [42] or the CY-BATHLON [43].…”

“…More specifically, the application of relevant and appropriate usability evaluation measures remains a fundamental challenge in wearable robotics development [19].…”

Challenges and solutions for application and wider adoption of wearable robots

“…Unfortunately, such brain measurements provide only limited information (e.g., left vs. right arm movement), and their utility as a stand-alone method is thus questionable except for severely paralyzed individuals (AL-Quraishi et al, 2018); however, they could potentially be combined with more popular modalities such as EMG (Li et al, 2019). As another example, heart rate, respiration and other physiological measures are commonly used as a metric of wearer workload (physical or mental) and thus robot performance (Torricelli et al, 2020), and could be used to adapt the amount of robot assistance; however, while studies have suggested complex approaches to doing this (Schürmann et al, 2019), only limited work in this area has been done, mostly with stationary wearable robots (e.g., the Lokomat) (Koenig et al, 2011a,b).…”

A Computational Framework Towards the Tele-Rehabilitation of Balance Control Skills

“…However, they have limitations in both their technology (e.g., sensor quality, resolution, processing power) and assessment approach (e.g., simple tracking of body motion and pressure distribution as proxies for the evaluation of balance control). Research efforts are being made to develop more accurate portable and wearable technologies for quantitative balance assessment ( Conforti et al, 2020 ; Torricelli et al, 2020 ) by including, for instance, inertial measurement units or electromyographic devices ( Zampogna et al, 2020 ).…”