Online, VR, AR, Lab, and In-Situ

RepliCueAuth: Validating the Use of a Lab-Based Virtual Reality Setup for Evaluating Authentication Systems

“…The contribution of this paper is threefold: (1) We propose the idea of using Virtual Reality as a test-bed for usability and security evaluations of real-world authentication systems. (2) We complement prior work that evaluated usability aspects in VR [64,104] by the first lab-based in-VR usability evaluation and the first online security evaluation through recordings in VR of a real-world authentication system and validate the use of VR through a comparison with the real-world study [52]. (3) Finally, we derive lessons learned to support researchers in designing, developing, and evaluating authentication systems of similar type in VR and discuss potential follow-up research directions.…”

“…While there are infrastructures that allow running online studies (e.g., Amazon Mechanical Turk) and result in valuable and inspiring privacy and security research [7,70,84,85], they are often not suitable for USEC research involving physical prototype systems. A promising emerging evaluation paradigm is Virtual Reality (VR) studies [64,89,104]. VR studies allow researchers to conduct evaluations by having participants experience virtual replicas of real-world systems in an immersive VR environment.…”

Investigating Effects and User Preferences of Extra- and Intradiegetic Virtual Reality Questionnaires

“…Additionally, our results suggest a general preference for in the experienced environment integrated questionnaires whatever the research method. Thus, our findings suggest relevance for all research methods [42], including online, lab and in-situ studies, which should be further researched in the future.…”

“…Further research could measure the effect of prior training, the implementation of a comment function desired by some participants, and other interaction forms than gesture-based inputs to mitigate technical challenged experienced in both our studies. Additionally, it should be explored to what extent our findings about IN VR-Qs apply to both usability testings and other research methods [42].…”

Special issue on pervasive displays

“…There are opportunities in experimenting with novel methodologies for evaluating pervasive displays. Recently, researchers have been investigating the use of virtual and augmented reality for evaluating ubiquitous systems, IoT devices, and security systems [14][15][16]. Most relevant to this special issue is the work of Mäkelä et al [14], which compared behavior of bystanders around public displays in the real world to their behavior around virtual displays in an immersive virtual environment.…”