How to Evaluate Object Selection and Manipulation in VR? Guidelines from 20 Years of Studies

Abstract

The VR community has introduced many object selection and manipulation techniques during the past two decades. Typically, they are empirically studied to establish their benefits over the stateof-the-art. However, the literature contains few guidelines on how to conduct such studies; standards developed for evaluating 2D interaction often do not apply. This lack of guidelines makes it hard to compare techniques across studies, to report evaluations consistently, and therefore to accumulate or replicate findings. To build such guidelines, we review 20 years of studies on VR object selection and manipulation. Based on the review, we propose recommendations for designing studies and a checklist for reporting them. We also identify research directions for improving evaluation methods and offer ideas for how to make studies more ecologically valid and rigorous. CCS CONCEPTS• Human-centered computing → HCI theory, concepts and models; HCI design and evaluation methods.

Communication in Immersive Social Virtual Reality: A Systematic Review of 10 Years' Studies

“…We conducted the systematic literature review following the PRISMA method [40]. PRISMA is a widely used method to ensure the reproducibility of the literature review in many disciplines, including human-computer interaction [6,34,59]. The PRISMA method contains four phases to identify eligible papers.…”

Evaluation of virtual handles for dental implant manipulation in virtual reality implant planning procedure

“…Translation, rotation, scaling, and selection are common actions in virtual environments [7,20]. Direct interaction includes grabbing the object, moving it around, and releasing the object.…”

Exploring the Effect of Personality Traits in VR Interaction: The Emergent Role of Perspective-Taking in Task Performance

“…Some examples include comparing the virtual hand with the virtual pointer metaphor (Poupyrev et al, 1998), exploring the effects of virtual hand representation on interaction with hand tracking input (Argelaguet et al, 2016), but also comparing it with controller-based methods (Masurovsky et al, 2020), and lastly studying different visual feedback approaches for virtual grasping (Canales et al, 2019). Performance is one of the main aspects of focus in these kinds of studies, measured mainly in terms of completion time and accuracy (Bergström et al, 2021). Additionally, they identify movement data, such as the path followed by the participants' hands, as the next most-although to a much lesser extent-popular type of measurement.…”

“…In their review study, Bergström et al (2021) formulated a set of guidelines for researchers planning object selection and manipulation studies in VR. Our study was designed and conducted before the release of their paper, and we had already applied very similar guidelines.…”

“…We explore two categories of personality traits, namely Empathy traits (Davis, 1983) and the Five Factor Model (FFM) traits (Costa and McCrae, 1992), and their effect on task performance through a series of object manipulation tasks. Object manipulation in VR is widely used for the assessment of interaction methods (Bergström et al, 2021). Our goal is to identify whether there are any concrete correlations with certain personality traits.…”