Predicting Goal-directed Attention Control Using Inverse-Reinforcement Learning

Abstract

Understanding how goal states control behavior is a question ripe for interrogation by new methods from machine learning. These methods require large and labeled datasets to train models. To annotate a large-scale image dataset with observed search fixations, we collected 16,184 fixations from people searching for either microwaves or clocks in a dataset of 4,366 images (MS-COCO). We then used this behaviorally-annotated dataset and the machine learning method of Inverse-Reinforcement Learning (IRL) to learn target-specific reward functions and policies for these two target goals. Finally, we used these learned policies to predict the fixations of 60 new behavioral searchers (clock = 30, microwave = 30) in a disjoint test dataset of kitchen scenes depicting both a microwave and a clock (thus controlling for differences in low-level image contrast). We found that the IRL model predicted behavioral search efficiency and fixation-density maps using multiple metrics. Moreover, reward maps from the IRL model revealed target-specific patterns that suggest, not just attention guidance by target features, but also guidance by scene context (e.g., fixations along walls in the search of clocks). Using machine learning and the psychologically-meaningful principle of reward, it is possible to learn the visual features used in goal-directed attention control.

Weighting the factors affecting attention guidance during free viewing and visual search: The unexpected role of object recognition uncertainty

“…For even greater specificity, we refer to priority maps in the context of a search task as “target maps” ( Zelinsky, 2008 ), a name that makes clear that the prioritization is based on a comparison of a visual input to features of a target goal. We adopted and consistently used this terminology in several recent studies ( Yang et al, 2020 ; Zelinsky et al, 2020 b), most clearly defined in Zelinsky and Bisley (2015 ), and we believe that these distinctions are particularly useful given our present goal of better understanding how different attention biases are weighted in the context of free-viewing and search tasks. In our view, when an assumption of top–down input is made, even in cases of simple text and face detection ( Liao, Shi, Bai, Wang, & Liu, 2017 ; Boyko, Basystiuk, & Shakhovska, 2018 ; Long, Ruan, Zhang, He, Wu, & Yao, 2018 ), a mixture of priority signals occurs that makes it challenging to compare models, with model performance often correlating with how much top–down input can be added to the prediction.…”