Towards More Robust Speech Interactions for Deaf and Hard of Hearing Users

Exploring Sound Awareness in the Home for People who are Deaf or Hard of Hearing

“…Sounds of interest: Similar to past work [2,8,13], the sounds of greatest interest were alarms and alerts (N=12), appliance timers (10), presence of other people and animals (9), and voices directed at you (7). In contrast, most participants (10) did not want to be aware of continuous background or mechanical noises unless the sound indicates a mechanical problem (e.g., a water leak) or emergency (e.g., a siren) (also identified in [2]).…”

“…For people with disabilities, smart home systems offer additional benefits [7,19]-for example, Pradhan et al [19] found that smart speakers, in some cases connected to smart home devices, can increase independence for users with visual and motor impairments. For DHH users, the recent introduction of tablet-based smart home technology (e.g., [25]) offers accessible alternatives to existing voicecontrolled systems, which have trouble understanding deaf speech [1,9]. Our work is complementary and could…”

“…Information overload: A sound awareness system could lead to overload if many sounds occur simultaneously. Indeed, in the movie scenario, where participants viewed sounds while pretending to work on a laptop, they felt overwhelmed (9) and distracted (6). Thus, nine participants wanted the system to filter sounds and gave suggestions for doing so.…”

Toward fairness in AI for people with disabilities SBG@a research roadmap

“…Today, many ASR systems do not work well for some older adults, due to differences in pitch, pacing, and clarity of speech by people of very advanced ages, since they are not commonly represented in the training and evaluation of the systems [67]. People with accents, including accents due to disability (e.g., "deaf accent"), also face challenges using current ASR tools [20,27,68], though it is possible to train personalized models for such groups [16,75]. Speech disabilities such as disarthrya, as well as the use of speech-generating augmentative and alternative communication (AAC) devices, can also negatively impact ASR functionality [38].…”

Understanding and Designing for Deaf or Hard of Hearing Drivers on Uber

“…However, because the recognition of Deaf speech using ASR is difficult, since the data is trained based on hearing individuals' speech, this new way of interaction does not seem accessible to DHH people. However, researchers [14,15] have begun analyzing the problem and have investigated the feasibility of improving the recognition of Deaf speech through ASR and iterative crowdsourcing workflows. Bigham et al [8] proposed two possible technical approaches: human-powered crowd sourcing workflows and making mobile speak for the DHH individual.…”