Conducting online virtual environment experiments with uncompensated, unsupervised samples

Huber, Beate; Gajos, Krzysztof Z.

doi:10.1371/journal.pone.0227629

Cited by 47 publications

(36 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Lastly, due to COVID-19 restrictions that prevented in-person laboratory testing, Experiment 2 was carried out online. Based on recent research indicating that online data collection can provide valid and reliable data on a variety of cognitive and perceptual experiments (Huber & Gajos, 2020 ; Komarov et al, 2013 ; Reinecke & Gajos, 2015 ), we expected to replicate the reversed congruency effects of Experiment 1 , in the condition without additional objects, as it was most similar to that of Experiment 1 .…”

Section: Methodsmentioning

confidence: 98%

Perspective taking and systematic biases in object location memory

Segen

Colombo²,

Avraamides

et al. 2021

Atten Percept Psychophys

View full text Add to dashboard Cite

The aim of the current study was to develop a novel task that allows for the quick assessment of spatial memory precision with minimal technical and training requirements. In this task, participants memorized the position of an object in a virtual room and then judged from a different perspective, whether the object has moved to the left or to the right. Results revealed that participants exhibited a systematic bias in their responses that we termed the reversed congruency effect. Specifically, they performed worse when the camera and the object moved in the same direction than when they moved in opposite directions. Notably, participants responded correctly in almost 100% of the incongruent trials, regardless of the distance by which the object was displaced. In Experiment 2, we showed that this effect cannot be explained by the movement of the object on the screen, but that it relates to the perspective shift and the movement of the object in the virtual world. We also showed that the presence of additional objects in the environment reduces the reversed congruency effect such that it no longer predicts performance. In Experiment 3, we showed that the reversed congruency effect is greater in older adults, suggesting that the quality of spatial memory and perspective-taking abilities are critical. Overall, our results suggest that this effect is driven by difficulties in the precise encoding of object locations in the environment and in understanding how perspective shifts affect the projected positions of the objects in the two-dimensional image.

show abstract

Section: Methodsmentioning

confidence: 98%

Perspective taking and systematic biases in object location memory

Segen

Colombo²,

Avraamides

et al. 2021

Atten Percept Psychophys

View full text Add to dashboard Cite

show abstract

“…The popular application VRchat has also been used to conduct supervised VR user studies (Saffo et al, 2020). The crowdsourcing platform LabInTheWild has also featured a desktop-VR study (Huber and Gajos, 2020).…”

Section: Crowdsourcingmentioning

confidence: 99%

Conducting Unsupervised Virtual Reality User Studies Online

Mottelson

Petersen

Lilija

et al. 2021

Front. Virtual Real.

View full text Add to dashboard Cite

Conducting user studies online and unsupervised instead of in laboratories gives quick access to a large and inexpensive participant pool. It is however unclear if data sourced this way is valid, and what the best practices for conducting unsupervised VR studies are. The restrictions on laboratory access experienced during COVID-19 further necessitate the development of valid procedures for remote data collection, especially for research fields such as VR that heavily rely on laboratory studies. In this paper we report our experiences with conducting two unsupervised VR studies amidst the pandemic, by recruiting participants online on relevant fora and employing participants’ own standalone VR equipment. We investigate whether it is feasible to collect valid data across in-VR survey responses and hand tracking. We report a good reliability of collected data, which requires only slightly more sanitation than a comparable laboratory study. We synthesize our experiences into practical recommendations for conducting unsupervised VR user studies using online recruitment, which can greatly reduce barriers to conducting empirical VR research and improve the quantity of VR user studies, regardless of laboratory availability.

show abstract

“…Mottelson et al [21] distributed Google Cardboard hardware to participants, while Steed et al [30] recruited Google Cardboard and Samsung Gear users directly via web pages and emails. To recruit VR participants more broadly, Huber et al [16] used a purpose-built scientific study platform (labinthewild.org). Ma et al [18] instead used Mechanical Turk directly.…”

Section: Related Workmentioning

confidence: 99%

“…The COVID-19 pandemic has only served to exacerbate these issues, as in-lab studies became ethically questionable and practically difficult in many locales. Existing research has examined various crowdsourcing methods for VR experiments [16,18,30]. But one potential method unique to VR has yet to be explored in detail -social VR platforms.…”

Section: Introductionmentioning

confidence: 99%

Remote and Collaborative Virtual Reality Experiments via Social VR Platforms

Saffo

Bartolomeo

Yıldırım

et al. 2021

Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems

View full text Add to dashboard Cite

Figure 1: Depiction of the two user studies we replicated using VRChat, a social VR platform. The darker robots represent the avatars of the participants. The lighter robots represent the avatars of the researchers, observing the participants and collecting data. Left: In quantitative Study 1 -extending Fitts' Law in 3D VR [8] -a participant selects a target by moving the cursor to intersect it. Right: In qualitative Study 2 -a trial from the tabletop study from Tang et al. [34] -two participants collaborate to find the best path between two nodes in a network.

show abstract

Conducting online virtual environment experiments with uncompensated, unsupervised samples

Cited by 47 publications

References 24 publications

Perspective taking and systematic biases in object location memory

Perspective taking and systematic biases in object location memory

Conducting Unsupervised Virtual Reality User Studies Online

Remote and Collaborative Virtual Reality Experiments via Social VR Platforms

Contact Info

Product

Resources

About