Kinematic markers of skill in first-person shooter video games

Abstract

Video games present a unique opportunity to study motor skill. First-person shooter (FPS) games have particular utility because they require visually-guided hand movements that are similar to widely studied planar reaching tasks. However, there is a need to ensure the tasks are equivalent if FPS games are to yield their potential as a powerful scientific tool for investigating sensorimotor control. Specifically, research is needed to ensure that differences in visual feedback of a movement do not affect motor learning between the two contexts. In traditional tasks, a movement will translate a cursor across a static background, whereas FPS games use movements to pan and tilt the view of the environment. To this end, we designed an online experiment where participants used their mouse or trackpad to shoot targets in both contexts. Kinematic analysis showed player movements were nearly identical between conditions, with highly correlated spatial and temporal metrics. This similarity suggests a shared internal model based on comparing predicted and observed displacement vectors, rather than primary sensory feedback. A second experiment, modelled on FPS-style aim-trainer games, found movements exhibited classic invariant features described within the sensorimotor literature. We found that two measures of mouse control, the mean and variability in distance of the primary sub-movement, were key predictors of overall task success. More broadly, these results show that FPS games offer a novel, engaging, and compelling environment to study sensorimotor skill, providing the same precise kinematic metrics as traditional planar reaching tasks.

Visuomotor memory is not bound to visual motion

“…To assess participants’ adaptation to perturbations in an FPS-style experiment, we utilized a framework allowing visual feedback to be compared between and within participants 15 . Participants used their mouse or trackpad (learning did not differ between input devices across all experiments, so is not considered hereafter) to control a cursor on the screen, appearing like an FPS-style aiming reticle.…”

“…For example, moving the mouse forward would cause the in-game character’s view to pan up . Critically, the two contexts were equated such that the same input movement in either context would reach a given target, allowing unbiased comparison of behavior between the contexts (see 15 for details).…”

“…While previous studies have observed long-term motor learning 62,64 , these typically focus on broad measures of improvement, for example the time required to roll cigars 65 . In contrast, the kinematic data collected can be used to decompose broad measures of improvement into a number of spatial and temporal metrics that concurrently improve and correlate well will more traditional Pointing movements 15 . We believe this, combined with our current findings, shows that both visual contexts predominantly tap into the same underlying processes, suggesting that findings from FPS games would generalize to the wider field.…”

“…In the main text we present data from both mouse and trackpad, as we did not observe any meaningful differences between these input devices. Experiments were created using a recently developed framework allowing within and between-participant comparison of Pointing and Looking movements 15 , which uses the Unity game engine (2019.4.15f) and the Unity Experiment Framework 72 . The experiments were delivered via a WebGL build hosted on a web page.…”

“…This lack of clarity on the characterization of internal model sensory predictions is felt when considering behaviors that differ only in terms of their visual feedback. We recently compared the same center-out reaching task in a mouse Pointing context, where a hand movement translates a cursor across a static background, to a first-person shooter (FPS) game inspired Looking visual context, where the same movement pans and tilts the view of the environment while the cursor remains central to the screen 15 . Movement kinematics and performance in these contexts were highly correlated within individuals, despite profound visual differences, suggesting that both contexts may be making use of the same internal model to plan and correct movements on the fly.…”