It is well known that help prompts shape how users talk to spoken dialogue systems. This study investigated the effect of help prompt placement on older users' interaction with a smart home interface. In the dynamic help condition, help was only given in response to system errors; in the inherent help condition, it was also given at the start of each task. Fifteen older and sixteen younger users interacted with a smart home system using two different scenarios. Each scenario consisted of several tasks. The linguistic style users employed to communicate with the system (interaction style) was measured using the ratio of commands to the overall utterance length (keyword ratio) and the percentage of content words in the user's utterance that could be understood by the system (shared vocabulary). While the timing of help prompts did not affect the interaction style of younger users, it was early task-specific help supported older users in adapting their interaction style to the system's capabilities. Well-placed help prompts can significantly increase the usability of spoken dialogue systems for older people.
User quality judgements can show a bewildering amount of variation that is difficult to capture using traditional quality prediction approaches. Using clustering, an exploratory statistical analysis technique, we reanalysed the data set of a Wizard-of-Oz experiment where 25 users were asked to rate the dialogue after each turn. The sparse data problem was addressed by careful a priori parameter choices and comparison of the results of different cluster algorithms. We found two distinct classes of users, positive and critical. Positive users were generally happy with the dialogue system, and did not mind errors. Critical users downgraded their opinion of the system after errors, used a wider range of ratings, and were less likely to rate the system positively overall. These user groups could not be predicted by experience with spoken dialogue systems, attitude to spoken dialogue systems, affinity with technology, demographics, or short-term memory capacity. We suggest that evaluation research should focus on critical users and discuss how these might be identified.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.