Conducting Unsupervised Virtual Reality User Studies Online

Abstract

Conducting user studies online and unsupervised instead of in laboratories gives quick access to a large and inexpensive participant pool. It is however unclear if data sourced this way is valid, and what the best practices for conducting unsupervised VR studies are. The restrictions on laboratory access experienced during COVID-19 further necessitate the development of valid procedures for remote data collection, especially for research fields such as VR that heavily rely on laboratory studies. In this paper we report our experiences with conducting two unsupervised VR studies amidst the pandemic, by recruiting participants online on relevant fora and employing participants’ own standalone VR equipment. We investigate whether it is feasible to collect valid data across in-VR survey responses and hand tracking. We report a good reliability of collected data, which requires only slightly more sanitation than a comparable laboratory study. We synthesize our experiences into practical recommendations for conducting unsupervised VR user studies using online recruitment, which can greatly reduce barriers to conducting empirical VR research and improve the quantity of VR user studies, regardless of laboratory availability.

A Separate Reality: An Update on Place Illusion and Plausibility in Virtual Reality

“…Some of these issues have been studied now for about three decades. With the entry of VR (and to a lesser extent AR) into the consumer market, there is the opportunity for conducting very large studies with participants from all walks of life (not just the typical student subjects), with experiences in their own homes (Steed et al, 2016;Mottelson et al, 2021). It also becomes even more pressing to address the vast amount of confusion there is in the literature.…”

Remote research on locomotion interfaces for virtual reality: Replication of a lab-based study on teleporting interfaces

“…Underrepresentation of women has been documented even in lab-based VR research [22], and this problem is likely to be compounded in remote VR research that recruits VR owners. In one recent and relatively large (n=226) remote VR study [17], women comprised only 12% of the final sample. Another potentially important sample difference is that VR headset owners, on average, spend more time playing video games than do non-owners [10].…”

Tracking motor activity in Virtual Reality to reveal cognitive functioning: A preliminary study

“…This study was conducted using a widely used commercial stand-alone headset. An advantage is that it provides standardisation in terms of hardware, which is likely to facilitate standardisation of experimental conditions when experimenting from a distance (Mottelson et al, 2021). We suggest that a follow-up to this preliminary study could be done using the SideQuest software (Steed et al, 2021).…”