Performance Evaluation Methods for Assistive Robotic Technology

Tsui, Katherine M.; Feil-Seifer, David; Matarić, Maja J.; Yanco, Holly A.

doi:10.1007/978-1-4419-0492-8_3

Cited by 18 publications

(7 citation statements)

References 69 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…When the wheelchair passed through the narrowest door, there was only a total of 10 cm in clearance. The ability to navigate through doorways without having a collision is both a common metric that researchers use to evaluate intelligent wheelchairs and a requirement in order to be prescribed a powered wheelchair in some countries [41].…”

Section: A Primary Taskmentioning

confidence: 99%

Collaborative Control for a Robotic Wheelchair: Evaluation of Performance, Attention, and Workload

Carlson

Demiris

2012

IEEE Trans. Syst., Man, Cybern. B

206

138

View full text Add to dashboard Cite

Abstract-Powered wheelchair users often struggle to drive safely and effectively and in more critical cases can only get around when accompanied by an assistant. To address these issues, we propose a collaborative control mechanism that assists the user as and when they require help. The system uses a multiple-hypotheses method to predict the driver's intentions and if necessary, adjusts the control signals to achieve the desired goal safely. The main emphasis of this paper is on a comprehensive evaluation, where we not only look at the system performance, but, perhaps more importantly, we characterise the user performance, in an experiment that combines eye-tracking with a secondary task. Without assistance, participants experienced multiple collisions whilst driving around the predefined route. Conversely, when they were assisted by the collaborative controller, not only did they drive more safely, but they were able to pay less attention to their driving, resulting in a reduced cognitive workload. We discuss the importance of these results and their implications for other applications of shared control, such as brain-machine interfaces, where it could be used to compensate for both the low frequency and the low resolution of the user input.

show abstract

Section: A Primary Taskmentioning

confidence: 99%

Collaborative Control for a Robotic Wheelchair: Evaluation of Performance, Attention, and Workload

Carlson

Demiris

2012

IEEE Trans. Syst., Man, Cybern. B

206

138

View full text Add to dashboard Cite

show abstract

“…Study results with inadequate, convenient or insufficiently described samples may not suffice to allow conclusions for persons with specific impairments which may represent the potential users of the RR. To ensure that RRs meet a user's needs and requirements and become successful on the market, it seems mandatory to involve the intended users at all stages of the design and evaluation process of such assistive robotic technologies [39,40,41]. …”

Section: Discussionmentioning

confidence: 99%

Evaluation Studies of Robotic Rollators by the User Perspective: A Systematic Review

Werner

Ullrich²,

Geravand³

et al. 2016

Gerontology

View full text Add to dashboard Cite

Background: Robotic rollators enhance the basic functions of established devices by technically advanced physical, cognitive, or sensory support to increase autonomy in persons with severe impairment. In the evaluation of such ambient assisted living solutions, both the technical and user perspectives are important to prove usability, effectiveness and safety, and to ensure adequate device application. Objective: The aim of this systematic review is to summarize the methodology of studies evaluating robotic rollators with focus on the user perspective and to give recommendations for future evaluation studies. Methods: A systematic literature search up to December 31, 2014, was conducted based on the Cochrane Review methodology using the electronic databases PubMed and IEEE Xplore. Articles were selected according to the following inclusion criteria: evaluation studies of robotic rollators documenting human-robot interaction, no case reports, published in English language. Results: Twenty-eight studies were identified that met the predefined inclusion criteria. Large heterogeneity in the definitions of the target user group, study populations, study designs and assessment methods was found across the included studies. No generic methodology to evaluate robotic rollators could be identified. We found major methodological shortcomings related to insufficient sample descriptions and sample sizes, and lack of appropriate, standardized and validated assessment methods. Long-term use in habitual environment was also not evaluated. Conclusions: Apart from the heterogeneity, methodological deficits in most of the identified studies became apparent. Recommendations for future evaluation studies include: clear definition of target user group, adequate selection of subjects, inclusion of other assistive mobility devices for comparison, evaluation of the habitual use of advanced prototypes, adequate assessment strategy with established, standardized and validated methods, and statistical analysis of study results. Assessment strategies may additionally focus on specific functionalities of the robotic rollators allowing an individually tailored assessment of innovative features to document their added value.

show abstract

“…The frequencies of both positive and negative behaviors are evaluated by either therapists or re-searchers. Post-hoc analysis of behavior observations from VDO sequences is a tedious task because therapists have to evaluate each of the image sequences ranging from tenths of a second to the entire session and the results depended on prior knowledge of each evaluator [6]. Automatic behavior understanding from images is thus required.…”

Section: Introductionmentioning

confidence: 99%

Repetitive motion detection for human behavior understanding from video images

Kumdee

Ritthipravat

2015

2015 IEEE International Symposium on Signal Processing and Information Technology (ISSPIT)

View full text Add to dashboard Cite

This paper aims to develop a technique for repetitive motion detection which is necessary for human behavior analysis particularly in children with autism spectrum disorders. Images from video sequences are mainly investigated. The technique uses image self-similarity measure, which is less sensitive to view changes, noise, and stable to low resolution images, as input data to multilayer perceptron neural network. Outputs of the network are composed of two classes, which are repetitive and non-repetitive motions. The classifier uses training data from a single person. The model is created by 10 fold cross validation. Trained network is tested with different data sets from seven normal subjects. The classification results show that the proposed technique provides an average accuracy of 0.9115 and can be used in real-time manner. In addition, the trained classifier is robust to images taken from different view.Keywords-repetitive motion detection; image self similarity measure; neural networks; children with autistic spectrum disorder.

show abstract

Performance Evaluation Methods for Assistive Robotic Technology

Cited by 18 publications

References 69 publications

Collaborative Control for a Robotic Wheelchair: Evaluation of Performance, Attention, and Workload

Collaborative Control for a Robotic Wheelchair: Evaluation of Performance, Attention, and Workload

Evaluation Studies of Robotic Rollators by the User Perspective: A Systematic Review

Repetitive motion detection for human behavior understanding from video images

Contact Info

Product

Resources

About