Virtual reality (VR) systems offer a powerful tool for human behavior research. The ability to create three-dimensional visual scenes and to measure responses to the visual stimuli enables the behavioral researcher to test hypotheses in a manner and scale that were previously unfeasible. For example, a researcher wanting to understand interceptive timing behavior might wish to violate Newtonian mechanics so that objects can move in novel 3-D trajectories. The same researcher might wish to collect such data with hundreds of participants outside the laboratory, and the use of a VR headset makes this a realistic proposition. The difficulty facing the researcher is that sophisticated 3-D graphics engines (e.g., Unity) have been created for game designers rather than behavioral scientists. To overcome this barrier, we have created a set of tools and programming syntaxes that allow logical encoding of the common experimental features required by the behavioral scientist. The Unity Experiment Framework (UXF) allows researchers to readily implement several forms of data collection and provides them with the ability to easily modify independent variables. UXF does not offer any stimulus presentation features, so the full power of the Unity game engine can be exploited. We use a case study experiment, measuring postural sway in response to an oscillating virtual room, to show that UXF can replicate and advance upon behavioral research paradigms. We show that UXF can simplify and speed up the development of VR experiments created in commercial gaming software and facilitate the efficient acquisition of large quantities of behavioral research data.

Video games present a unique opportunity to study motor skill. First-person shooter (FPS) games have particular utility because they require visually-guided hand movements that are similar to widely studied planar reaching tasks. However, there is a need to ensure the tasks are equivalent if FPS games are to yield their potential as a powerful scientific tool for investigating sensorimotor control. Specifically, research is needed to ensure that differences in visual feedback of a movement do not affect motor learning between the two contexts. In traditional tasks, a movement will translate a cursor across a static background, whereas FPS games use movements to pan and tilt the view of the environment. To this end, we designed an online experiment where participants used their mouse or trackpad to shoot targets in both contexts. Kinematic analysis showed player movements were nearly identical between conditions, with highly correlated spatial and temporal metrics. This similarity suggests a shared internal model based on comparing predicted and observed displacement vectors, rather than primary sensory feedback. A second experiment, modelled on FPS-style aim-trainer games, found movements exhibited classic invariant features described within the sensorimotor literature. We found that two measures of mouse control, the mean and variability in distance of the primary sub-movement, were key predictors of overall task success. More broadly, these results show that FPS games offer a novel, engaging, and compelling environment to study sensorimotor skill, providing the same precise kinematic metrics as traditional planar reaching tasks.

Some activities can be meaningfully dichotomised as ‘cognitive’ or ‘sensorimotor’ in nature—but many cannot. This has radical implications for understanding activity limitation in disability. For example, older adults take longer to learn the serial order of a complex sequence but also exhibit slower, more variable and inaccurate motor performance. So is their impaired skill acquisition a cognitive or motor deficit? We modelled sequence learning as a process involving a limited capacity buffer (working memory), where reduced performance restricts the number of elements that can be stored. To test this model, we examined the relationship between motor performance and sequence learning. Experiment 1 established that older adults were worse at learning the serial order of a complex sequence. Experiment 2 found that participants showed impaired sequence learning when the non-preferred hand was used. Experiment 3 confirmed that serial order learning is impaired when motor demands increase (as the model predicted). These results can be captured by reinforcement learning frameworks which suggest sequence learning will be constrained both by an individual’s sensorimotor ability and cognitive capacity.

Consumer virtual reality (VR) systems are increasingly being deployed in research to study sensorimotor behaviors, but properties of such systems require verification before being used as scientific tools. The ‘motion-to-photon’ latency (the lag between a user making a movement and the movement being displayed within the display) is a particularly important metric as temporal delays can degrade sensorimotor performance. Extant approaches to quantifying this measure have involved the use of bespoke software and hardware and produce a single measure of latency and ignore the effect of the motion prediction algorithms used in modern VR systems. This reduces confidence in the generalizability of the results. We developed a novel, system-independent, high-speed camera-based latency measurement technique to co-register real and virtual controller movements, allowing assessment of how latencies change through a movement. We applied this technique to measure the motion-to-photon latency of controller movements in the HTC Vive, Oculus Rift, Oculus Rift S, and Valve Index, using the Unity game engine and SteamVR. For the start of a sudden movement, all measured headsets had mean latencies between 21 and 42 ms. Once motion prediction could account for the inherent delays, the latency was functionally reduced to 2–13 ms, and our technique revealed that this reduction occurs within ~25–58 ms of movement onset. Our findings indicate that sudden accelerations (e.g., movement onset, impacts, and direction changes) will increase latencies and lower spatial accuracy. Our technique allows researchers to measure these factors and determine the impact on their experimental design before collecting sensorimotor data from VR systems.

experiments created in commercial gaming software and facilitate the efficient 36 acquisition of large quantities of behavioural research data. 37