Multidimensional evaluation of Virtual Reality paradigms in clinical neuropsychology: the VR-Check framework

Abstract

Virtual Reality (VR) represents a key technology of the 21st century, attracting substantial interest from a wide range of scientific disciplines. With regard to clinical neuropsychology, a multitude of new VR applications are being developed to overcome the limitations of classical paradigms. In consequence, researchers increasingly face the challenge of systematically evaluating the characteristics and quality of VR applications in order to design the optimal paradigm for their specific research question and study population. However, the manifold properties of contemporary VR are not adequately captured by the psychometric quality criteria of classical test theory (i.e., objectivity, reliability, validity), highlighting the need for an extended paradigm evaluation framework. To address this gap, we here propose a multidimensional evaluation framework for VR applications in clinical neuropsychology, summarized as an easy-to-use checklist (VR-Check). This framework rests on ten main evaluation dimensions encompassing cognitive domain specificity, ecological relevance, technical feasibility, user feasibility, user motivation, task adaptability, performance quantification, immersive capacities, training feasibility, and predictable pitfalls. We show how VR-Check enables systematic and comparative paradigm optimization by illustrating its application in an exemplary research project on the assessment of spatial cognition and executive functions with immersive VR. This application furthermore demonstrates how the framework allows researchers to identify across-domain tradeoffs, makes deliberate design decisions explicit, and optimizes the allocation of study resources. Complementing recent approaches to standardize clinical VR studies, the VR-Check framework enables systematic and project-specific paradigm optimization for behavioral and cognitive research in neuropsychology.

Automated Stress Recognition Using Supervised Learning Classifiers by Interactive Virtual Reality Scenes

“…The software that encompassed diverse scenarios pertaining to different types of stressors took over a year to develop; the stressors include distraction-induced stress in Snow Valley Adventure, feared-stimuli-based stress in Scary Monsters, and bonus-penalty-stimuli-based Enemy Shooting, which promoted user feasibility and motivation during task completion. Such designs are in line with recent multidimensional evaluations of VR paradigms in behavior research [48].…”

Ethical Challenges of Using Virtual Environments in the Assessment and Treatment of Psychopathological Disorders

“…While there are increasing efforts aimed at psychometric validation of virtual reality based neuropsychological assessments [88][89][90], clinicians must use their ethical judgements to balance the added understanding of the patient's performance of activities of daily living (from the VE) with the lack of adequate norms. Before virtual environments can be widely adopted, there is need for focused guidelines on the development (design issues and manuals), norming, psychometric validation, and administration of these VE platforms [91][92][93]. From an ethical perspective, much more research is needed before clinicians can rely on virtual environments for their assessments.…”

Assessing Goal-oriented Behavior in Virtual Reality: Children Prefer Head-mounted Displays but Flat Screens Provide a Viable Performance Measure for Remote Testing

“…The literature of virtual reality (VR) based cognition research is expanding at a rapid pace, reflecting the increasing availability of VR systems and their technological advancements (Cipresso et al, 2018;Krohn et al, 2020). Notably, VR has been suggested as an ideal way of implementing new naturalistic paradigms that mimic real-life functions and situations (Chan et al, 2008;Parsons et al, 2017), as it offers safe and flexible ways to create various easily-reproducible environments and allows diverse behavioral responses (e.g., movements of the eyes, head, and body) to be measured accurately (see Campbell et al, 2009).…”