Limited linguistic coverage for Intelligent Personal Assistants (IPAs) means that many interact in a non-native language. Yet we know little about how IPAs currently support or hinder these users. Through native (L1) and non-native (L2) English speakers interacting with Google Assistant on a smartphone and smart speaker, we aim to understand this more deeply. Interviews revealed that L2 speakers prioritised utterance planning around perceived linguistic limitations, as opposed to L1 speakers prioritising succinctness because of system limitations. L2 speakers see IPAs as insensitive to linguistic needs resulting in failed interaction. L2 speakers clearly preferred using smartphones, as visual feedback supported diagnoses of communication breakdowns whilst allowing time to process query results. Conversely, L1 speakers preferred smart speakers, with audio feedback being seen as sufficient. We discuss the need to tailor the IPA experience for L2 users, emphasising visual feedback whilst reducing the burden of language production. CCS Concepts • Human-centered computing → User studies; Natural language interfaces; Accessibility design and evaluation methods.
Through smartphones and smart speakers, intelligent personal assistants (IPAs) have made speech a common interaction modality. With linguistic coverage and varying functionality levels, many speakers engage with IPAs using a non-native language. This may impact mental workload and patterns of language production used by non-native speakers. We present a mixed-design experiment, where native (L1) and non-native (L2) English speakers completed tasks with IPAs via smartphones and smart speakers. We found significantly higher mental workload for L2 speakers in IPA interactions. Contrary to our hypotheses, we found no significant differences between L1 and L2 speakers in number of turns, lexical complexity, diversity, or lexical adaptation when encountering errors. These findings are discussed in relation to language production and processing load increases for L2 speakers in IPA interaction. CCS CONCEPTS • Human-centered computing → User studies; Natural language interfaces; HCI theory, concepts and models.
Speech-enabled smart speakers are common devices used for numerous tasks in everyday life. While speech-enabled technologies are widespread, using one's voice as a computing modality introduces new accessibility challenges for people with speech disfluencies such as stammering (also known as stuttering). This paper investigates the smart speaker user experiences of people who stammer over three weeks. We conducted diary studies and semi-structured interviews with 11 individuals to identify their daily routines, difficulties with successful interactions, and strategies to overcome these barriers. Our analysis demonstrates key factors such as device location, its affordances, and the structure of commands had a strong impact on user experience. Participants highlighted different linguistic strategies to try and overcome interaction difficulties and discussed the potential of using smart speakers for speech and language therapy. We emphasise the need to further understand the experiences of people who stammer in smart speaker design to increase their accessibility.CCS Concepts: • Human-centered computing → Empirical studies in accessibility; User studies; Empirical studies in HCI.
Figure 1: We developed a room scale virtual reality game where players controlled toy army men using speech and gaze. The environment, shown above, included a range of game objects and everyday objects distributed in the environment that could be referenced by deictic expressions or name. The toy soldier agents could be commanded to complete tasks such as occupy a checkpoint, capture a flag, or interact with objects in the environment.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.