Abstract. Mobile technologies such as multimedia guides (MMGs) are now an important part of the visitor experience in museums and other cultural spaces. We report the development of two scales to measuring visitors' museum experiences: the Museum Experience Scale (MES) and the Multimedia Guide Scale (MGS); these quantitative measures can helpfully complement qualitative information about visitor experience. A standard psychometric methodology was used in the development of these scales: from a large set of potentially relevant statements, 57 were chosen and 255 people rated a museum experience (102 of whom had used a multimedia guide). A Principal Components analysis yielded a four factor solution for the MES (Engagement, Knowledge/Learning, Meaningful Experience and Emotional Connection) and a three factor solution for the MMGS (General Usability, Learnability and Control, Quality of Interaction). Comparing respondents who used a MMG during their museum visit with those who did not, there was a significant difference on the Engagement component of the MES, with respondents who used a MMG being significantly more engaged. The other components of the MES did not show significant differences.

This paper reports on an empirical study that compares two sets of heuristics, Nielsen’s heuristics and the SMART heuristics in the identification of usability problems in a mobile guide smartphone app for a living museum. Five experts used the severity rating scales to identify and determine the severity of the usability issues based on the two sets of usability heuristics. The study found that Nielsen’s heuristics set is too general to detect usability problems in a mobile application compared to SMART heuristics which focuses on the smartphone application in the product development lifecycle instead of the generic Nielsen’s heuristics which focuses on a wide range of interactive system. The study highlights the importance of utilizing domain specific usability heuristics in the evaluation process. This ensures that relevant usability issues were successfully identified which could then be given immediate attention to ensure optimal user experience.

This paper elaborates the empirical evidence of a usability evaluation of a VR and non-VR virtual tour application for a living museum. The System Usability Scale (SUS) was used in between participants experiments (Group 1: non-VR version and Group 2: VR version) with 40 participants. The results show that the mean scores of all components for the VR version are higher compared to the non-VR version, overall SUS score (72.10 vs 68.10), usability score (75.50 vs 71.70), and learnability (58.40 vs 57.00). Further analysis using a two-tailed independent t test showed no difference between the non-VR and VR versions. Additionally, no significant difference was observed between the groups in the context of gender, nationality, and prior experience (other VR tour applications) for overall SUS score, usability score, and learnability score. Α two-tailed independent t test indicated no significant difference in the usability score between participants with VR experience and no VR experience. However, a significant difference was found between participants with VR experience and no VR experience for both SUS score (t(38) = 2.17, p = 0.037) and learnability score (t(38) = 2.40, p = 0.021). The independent t test results indicated a significant difference between participant with and without previous visits to SCV for the usability score (t(38) = −2.31, p = 0.027), while there was no significant differences observed in other components. It can be concluded that both versions passed based on the SUS score. However, the sub-scale usability and learnability scores indicated some usability issue.

No abstract

Smartphone technologies are now used to deploy audio and multimedia guides in cultural spaces including historic churches. It is important to measure what effect the use of such technologies have on visitor experience of the cultural space and the usability and user experience of the guide. An "in the wild" study was conducted to investigate visitor experience in a historic church, with two versions of a multimedia iPhone Guide and with a traditional paper guide. The Church Experience Scale (CES) and the Multimedia Guide Scale (MMGS) were used to measure experience of 59 visitors to Holy Trinity Church, Stratfordupon-Avon, known as Shakespeare's church. 40 visitors used an iPhone guide, 21 a free choice version and 19 a guided tour version and 19 visitors used a paper guide. Results showed that participants with a smartphone guide had a significantly more positive visitor experience and spent significantly longer on their visit to the church. There was a significant correlation between length of visit and the Enjoyment, Intellectual Stimulation and Curiosity factor of the CES, but further work is needed to understand the direction of causality in this relationship. The usability and user experience of the multimedia guide conditions did not differ from each other, but the results of the MMGS showed that both guides could be improved in terms of general usability and quality of interaction. Challenges of conducting a study "in the wild" of a smartphone app in a cultural space are discussed.

No abstract

This empirical study was conducted to design, develop, and evaluate children's experiences with a game-based mobile guide (GBMG) application at Sarawak Cultural Village (SCV). The Interaction Design lifecycle model was used for the systematic development of the GBMG application. A total of 45 children took part in this study who were divided into two groups: a paper-based pamphlet and GBMG application group. The Museum Experience Scale were used to evaluate the overall experience with the GBMG application in this study. Results showed that the children in the mobile-based groups have a higher mean for the four dimensions of MES, although the difference is only significant for the emotional connection dimension. This study shows that the game-based mobile guide application did not significantly improve children's museum experience at SCV except for their emotional connection. Additional findings from the Heuristic Evaluation with six Human-Computer Interaction experts offered a deeper understanding of why the GBMG did not improve the children's visitor experience at SCV. The outcome of this study contributes to the research field of game-based mobile applications to enhance children's experiences at living museums with several issues raised for further research.

Providing quality and meaningful experiences to the users are vital particularly with the devices that users are familiar with. Usability issues should be discarded before the systems/designs are deployed to the users. In this study, we compared the usability issues found by expert and novice users using Collaborative Heuristic Evaluation (CHE). This study was carried out with 6 experts and 9 novice users. There are 3 different mobile guides used in this study, 2 were designed to be used in museum and another mobile guide was designed to be used at a tourist attraction. Results showed that expert users found significantly more heuristic problems than novice users during the CHE procedure across the 3 different mobile guides (smartphone apps). On the other hand, there were heuristic problems found by the novices that were not noticeable to the expert.