USE: An integrative suite for temporally-precise psychophysical experiments in virtual environments for human, nonhuman, and artificially intelligent agents

Watson, Marcus R.; Voloh, Benjamin; Corey, Thomas; Hasan, Asif; Womelsdorf, Thilo

doi:10.1016/j.jneumeth.2019.108374

Cited by 43 publications

(59 citation statements)

References 41 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The experiment was controlled by USE (Unified Suite for Experiments) using the Unity 3D game engine for behavioral control and visual display (Watson et al, 2019b). Four animals performed the experiment in cage-based touchscreen Kiosk Testing Station described in (Womelsdorf et al, in preparation), while two animals performed the experiment in a sound attenuating experimental booth.…”

Section: Methodsmentioning

confidence: 99%

Learning at variable attentional load requires cooperation between working memory, meta-learning and attention-augmented reinforcement learning

Womelsdorf

Watson

Tiesinga³

2020

Preprint

Self Cite

View full text Add to dashboard Cite

Flexible learning of changing reward contingencies can be realized with different strategies. A fast learning strategy involves forming a working memory of rewarding experiences with objects to improve future choices. A slower learning strategy uses prediction errors to gradually update value expectations to improve choices. How the fast and slow strategies work together in scenarios with real-world stimulus complexity is not well known. Here, we disentangle their relative contributions in rhesus monkeys while they learned the relevance of object features at variable attentional load. We found that learning behavior across six subjects is consistently best predicted with a model combining (i) fast working memory (ii) slower reinforcement learning from positive and negative prediction errors, as well as (iii) selective suppression of non-chosen features values, and (iv) meta-learning based adjustment of exploration rates given a memory trace of recent errors. These mechanisms cooperate differently at low and high attentional loads. While working memory was essential for efficient learning at lower attentional loads, learning from negative outcomes and metalearning were essential for efficient learning at higher attentional loads. Together, these findings highlight that a canonical set of distinct learning mechanisms cooperate to optimize flexible learning when adjusting to environments with real-world attentional demands.

show abstract

Section: Methodsmentioning

confidence: 99%

Learning at variable attentional load requires cooperation between working memory, meta-learning and attention-augmented reinforcement learning

Womelsdorf

Watson

Tiesinga³

2020

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…This intelligent visual search and selective attention is constrained by the “bandwidth” of both the visual media as well as the learners themselves (in terms of efficiency of visual processing). The digital media and VR have offered excellent means to simulate the richer learning environment and, hence, to improve the practice of scientific experiments and learning under peer pressure with appropriate social interactions (Watson, Voloh, Thomas, Hasan, & Womelsdorf, ; Zhou, Han, Liang, Hu, & Kuai, ). However, the basic visual processing principles could be fundamentally updated within the new type of spaces and temporal contexts to better address the new learning styles.…”

Section: Limitations and Outstanding Questionsmentioning

confidence: 99%

Education and visual neuroscience: A mini‐review

Chen

2019

PsyCh Journal

View full text Add to dashboard Cite

Neuroscience, especially visual neuroscience, is a burgeoning field that has greatly shaped the format and efficacy of education. Moreover, findings from visual neuroscience are an ongoing source of great progress in pedagogy. In this mini‐review, I review existing evidence and areas of active research to describe the fundamental questions and general applications for visual neuroscience as it applies to education. First, I categorize the research questions and future directions for the role of visual neuroscience in education. Second, I juxtapose opposing views on the roles of neuroscience in education and reveal the “neuromyths” propagated under the guise of educational neuroscience. Third, I summarize the policies and practices applied in different countries and for different age ranges. Fourth, I address and discuss the merits of visual neuroscience in art education and of visual perception theories (e.g., those concerned with perceptual organization with respect to space and time) in reading education. I consider how vision‐deprived students could benefit from current knowledge of brain plasticity and visual rehabilitation methods involving compensation from other sensory systems. I also consider the potential educational value of instructional methods based on statistical learning in the visual domain. Finally, I outline the accepted translational framework for applying findings from educational neuroscience to pedagogical theory.

show abstract

“…Horstmann et al (2019) found that the average number of fixations on visual targets (about 1.55) was higher compared to the average number of fixations on similar looking distractors (about 1.20) during a search task with static images. Watson et al (2019) reported that the number of fixations on targets ranged from about 3.3 to 4 compared to around 2.8 to 3.8 fixations on distractors, during a free visual search and reward learning task in a virtual environment. In terms of dwell time, Draschkow et al (2014) found subjects looked about 0.6 seconds longer at targets as compared to distractors during visual search of static natural scenes.…”

Section: Introductionmentioning

confidence: 99%

“…Since eye-tracking systems can now be readily integrated with 3D rendering software (i.e. game engines), researchers can conduct eye movement studies in more realistic and immersive environments (Watson et al, 2019). Virtual environments also allow for research designs that may otherwise not be practical for a real-world implementation.…”

Section: Introductionmentioning

confidence: 99%

Identification of Target Objects from Gaze Behavior during a Virtual Navigation Task

Enders¹,

Smith²,

Gordon³

et al. 2021

Preprint

View full text Add to dashboard Cite

Eye tracking has been an essential tool within the vision science community for many years. However, the majority of studies involving eye-tracking technology employ a relatively passive approach through the use of static imagery, prescribed motion, or video stimuli. This is in contrast to our everyday interaction with the natural world where we navigate our environment while actively seeking and using task-relevant visual information. For this reason, vision researchers are beginning to use virtual environment platforms, which offer interactive, realistic visual environments while maintaining a substantial level of experimental control. Here, we recorded eye movement behavior while participants freely navigated through a complex virtual environment. Within this environment, participants completed a visual search task where they were asked to find and count occurrence of specific targets among numerous distractor items. We assigned each participant into one of four target groups: Humvees, motorcycles, aircraft, or furniture. Our results show a significant relationship between gaze behavior and target objects across subject groups. Specifically, we see an increased number of fixations and increase dwell time on target relative to distractor objects. In addition, we included a divided attention task to investigate how search patterns changed with the addition of a secondary task. With increased cognitive load, subjects slowed their speed, decreased gaze on objects, and increased the number of objects scanned in the environment. Overall, our results confirm previous findings from more controlled laboratory settings and demonstrate that complex virtual environments can be used for active visual search experimentation, maintaining a high level of precision in the quantification of gaze information and visual attention. This study contributes to our understanding of how individuals search for information in a naturalistic virtual environment. Likewise, our paradigm provides an intriguing look into the heterogeneity of individual behaviors when completing an un-timed visual search task while actively navigating.

show abstract

USE: An integrative suite for temporally-precise psychophysical experiments in virtual environments for human, nonhuman, and artificially intelligent agents

Abstract: This document describes the tests performed to characterize USE system latencies relating to the USE I/O Box. Test methods and results are summarized.

Cited by 43 publications

References 41 publications

Learning at variable attentional load requires cooperation between working memory, meta-learning and attention-augmented reinforcement learning

Learning at variable attentional load requires cooperation between working memory, meta-learning and attention-augmented reinforcement learning

Education and visual neuroscience: A mini‐review

Identification of Target Objects from Gaze Behavior during a Virtual Navigation Task

Contact Info

Product

Resources

About