Alternative questionnaire formats in usability evaluation

Abstract

This doctoral thesis investigated the use of image-based scales in usability assessment, examining their advantages and limitations. While previous research attributed favourable qualities to image-based scales, such as increased motivation, intuitive comprehension, reduced workload, and reduced completion times, these claims often lacked solid empirical support. This work aimed to systematically evaluate variations of image-based scales in relation to these respondent-centred aspects, which are subsumed under the concept of questionnaire experience (QX). Furthermore, traditional psychometric properties were assessed. Three studies were conducted in which image-based scales were used in an online usability test setting. Study one used a hybrid version (i.e. pictorial and verbal content) of an existing usability questionnaire. Study two compared hybrid and purely pictorial scales in short and long versions. Study three introduced an animated hybrid scale. The findings from the three studies suggest that pictorial, hybrid and animated scales demonstrated satisfactory psychometric properties, making them viable alternatives for assessing perceived usability. However, QX between pictorial and hybrid scales differed considerably. Hybrid instruments received higher ratings on respondent-centred aspects and were more preferred. Furthermore, the findings highlight that some of the favourable qualities of image-based scales appeared to be too optimistic (e.g. reduced completion times). Nevertheless, the notion of increased motivation when using image-based scales could be largely supported. Theoretical and practical implications are discussed.