Investigating the Accessibility of Voice Assistants With Impaired Users: Mixed Methods Study (Preprint)

Abstract

BACKGROUND Voice assistants allow users to control appliances and functions of a domotic house by simply uttering a few words. Such systems hold the potential to significantly help users with motor and cognitive disabilities who currently depend on their caregiver even for basic needs, e.g., opening a door. The research on voice assistants is mainly dedicated to abled-bodied users, and studies evaluating the accessibility of such systems are still sparse and fail to account for the participants’ actual cognitive and linguistic abilities. OBJECTIVE In the present work, we aimed to investigate whether cognitive and/or linguistic functions could predict the user’s performance in operating an off-the-shelf voice assistant, namely Google Home. METHODS A group of users with disabilities (N=16) was involved in a living laboratory and was asked to interact with the system. Besides collecting data regarding their performance and their experience with the system, we assessed their cognitive and linguistic skills using standardized inventories. The identification of predictors (cognitive and/or linguistic) capable to account for an efficient interaction with the voice assistant was investigated by performing multiple linear regression models. The best model was identified by adopting a selection strategy based on the Akaike Information Criterion (AIC). RESULTS The best model (AIC = 127.81) shows that, among several cognitive and linguistic predictors, the Mini-Mental State Examination (MMSE) and the ability to repeat sentences (Robertson Dysarthria Profile) were crucial to predict the user’s performance. CONCLUSIONS Our findings show that cognitive and linguistic skills are involved and co-participate in the effective interaction with voice assistants. Furthermore, the present study provides design implications based on the mistakes observed during the interaction with voice assistants and advances practical indicators to predict the level of accessibility of voice assistants.

You have interrupted me again!: making voice assistants more dementia-friendly with incremental clarification

“…Voice assistant accessibility is therefore critical to ensure future systems are designed with the end user's interaction patterns and needs in mind (Ballati et al, 2018;Brewer et al, 2018;Addlesee, 2023). Industry voice assistants are created for the mass market (Masina et al, 2020), and today they are found in our homes, our cars, and many of our pockets (Ballati et al, 2018). They are trained on vast data sets to learn how to interact with the 'average user' , but speech production is nuanced, and not everyone perceives the world in the same way (Masina et al, 2020(Masina et al, , 2021Addlesee, 2023).…”

“…Industry voice assistants are created for the mass market (Masina et al, 2020), and today they are found in our homes, our cars, and many of our pockets (Ballati et al, 2018). They are trained on vast data sets to learn how to interact with the 'average user' , but speech production is nuanced, and not everyone perceives the world in the same way (Masina et al, 2020(Masina et al, , 2021Addlesee, 2023). For example, certain user groups-often those who can benefit the most from voice assistants-speak more disfluently than the average user (Addlesee et al, 2019;Masina et al, 2020;Ehghaghi et al, 2022).…”

“…Dementia is the leading cause of death in the United Kingdom, but there is no treatment to prevent, cure, or stop its progression (Alzheimer's Research UK, 2022). It impacts memory, attention, problem-solving skills, decisionmaking, speech production (Slegers et al, 2018;Masina et al, 2020), and more (Rudzicz et al, 2015;Association, 2019;Li et al, 2020). Onset and progression of cognitive impairment typically correlates with a person's age, but certain conditions (e.g., early-onset et al, 2023).…”

“…Over 1 billion people in the world are living with some form of disability (WHO, 2011;Domingo, 2012;Vieira et al, 2022), and everyday voice assistants (EVAs) have the potential to improve people's lives (Pradhan et al, 2018;Shalini et al, 2019;Masina et al, 2020). Household open-domain voice assistants are very convenient: we can set timers when our hands are oily from cooking or turn up our music from the comfort of a warm blanket on the couch.…”

A Novel Approach Using Flutter for Voice Controlled Assistant

“…result of the same new age technology called AI. At the core, these assistants is a simple synchronous cycle includes Voice commands [4] and hear responses.…”