Usability evaluation of a virtual reality smartphone app for a living museum

Abstract

This paper elaborates the empirical evidence of a usability evaluation of a VR and non-VR virtual tour application for a living museum. The System Usability Scale (SUS) was used in between participants experiments (Group 1: non-VR version and Group 2: VR version) with 40 participants. The results show that the mean scores of all components for the VR version are higher compared to the non-VR version, overall SUS score (72.10 vs 68.10), usability score (75.50 vs 71.70), and learnability (58.40 vs 57.00). Further analysis using a two-tailed independent t test showed no difference between the non-VR and VR versions. Additionally, no significant difference was observed between the groups in the context of gender, nationality, and prior experience (other VR tour applications) for overall SUS score, usability score, and learnability score. Α two-tailed independent t test indicated no significant difference in the usability score between participants with VR experience and no VR experience. However, a significant difference was found between participants with VR experience and no VR experience for both SUS score (t(38) = 2.17, p = 0.037) and learnability score (t(38) = 2.40, p = 0.021). The independent t test results indicated a significant difference between participant with and without previous visits to SCV for the usability score (t(38) = −2.31, p = 0.027), while there was no significant differences observed in other components. It can be concluded that both versions passed based on the SUS score. However, the sub-scale usability and learnability scores indicated some usability issue.

A Multidisciplinary Telerehabilitation Approach for Supporting Social Interaction in Autism Spectrum Disorder Families: An Italian Digital Platform in Response to COVID-19

“…The overall mean learnability score (61.7) suggested that this component was deemed only "fair" by users, so further studies are necessary to investigate this factor. However, in accordance with similar usability tests [37], this outcome is likely related to the participants not being given specific instructions on how to use the different tools.…”

“…We obtained similar results by analyzing the two different user groups. Alongside a unidimensional analysis of the SUS score, we conducted a bidimensional analysis, evaluating the usability and learnability subscales in accordance with Othman et al [ 37 ]. With reference to the usability subscale, the mean score obtained (96.1) suggested there were no issues in the usability system; in fact, all of the participants, both therapists (97.1) and parents (94.6), described SUPER as very user friendly.…”

“…More specifically, following the adjective grade ranking proposed by Bangor et al [ 36 ], scores over 80 suggest an “excellent” usable system, scores between 68 and 80 can be considered “good”, scores between 51 and 68 indicate “fair” results, scores below 51 are “poor”, and scores below 36 reflect “unusable” systems. In addition, since our participants were experts in the application domains [ 37 ], we also performed a bidimensional analysis of the SUS scale, involving the usability and learnability subscales [ 38 ]. The usability subscale covers items 1, 2, 3, 5, 6, 7, 8, and 9 from the SUS scale, while items 4 and 10 are included in the learnability subscale.…”

Usability Evaluation of Web Search User Interfaces from the Elderly Perspective

“…The system usability scale (SUS) is a popular tool among HCI researchers for assessing perceived usability in both usability and survey research [30,31,32]. According to reports, the SUS was used in 43% of usability studies [33,34]. Furthermore, a study that evaluated SUS discovered a valid and reliable tool for evaluating usability [34,35].…”

“…According to reports, the SUS was used in 43% of usability studies [33,34]. Furthermore, a study that evaluated SUS discovered a valid and reliable tool for evaluating usability [34,35]. The SUS is a ten items questionnaire with alternating positive and negative statements and a five-point Likert scale spanning from strongly disagree to strongly agree in ascending order designed to avoid answer biases.…”