Atari-HEAD: Atari Human Eye-Tracking and Demonstration Dataset

Abstract

Large-scale public datasets have been shown to benefit research in multiple areas of modern artificial intelligence. For decision-making research that requires human data, high-quality datasets serve as important benchmarks to facilitate the development of new methods by providing a common reproducible standard. Many human decision-making tasks require visual attention to obtain high levels of performance. Therefore, measuring eye movements can provide a rich source of information about the strategies that humans use to solve decision-making tasks. Here, we provide a large-scale, high-quality dataset of human actions with simultaneously recorded eye movements while humans play Atari video games. The dataset consists of 117 hours of gameplay data from a diverse set of 20 games, with 8 million action demonstrations and 328 million gaze samples. We introduce a novel form of gameplay, in which the human plays in a semi-frame-by-frame manner. This leads to near-optimal game decisions and game scores that are comparable or better than known human records. We demonstrate the usefulness of the dataset through two simple applications: predicting human gaze and imitating human demonstrated actions. The quality of the data leads to promising results in both tasks. Moreover, using a learned human gaze model to inform imitation learning leads to an 115% increase in game performance. We interpret these results as highlighting the importance of incorporating human visual attention in models of decision making and demonstrating the value of the current dataset to the research community. We hope that the scale and quality of this dataset can provide more opportunities to researchers in the areas of visual attention, imitation learning, and reinforcement learning.

Reinforcement Learning Interpretation Methods: A Survey

“…An interesting aspect about this approach is that attention is learnt from humans by utilizing the eye tracking data and instead of just utilizing it for evaluation like the previous method [60]. Nikuli et al [2] suggested a quantitative method to evaluate the attention mechanism generated saliency maps by comparing them with the eye tracking data publicly available by [62]. They used their proposed metric to compare different attention based RL models.…”

Leveraging Human Guidance for Deep Reinforcement Learning Tasks

“…Human visual attention data is often obtained using eyetrackers to record their eye movements (gaze). Recently, researchers have collected large-scale human gaze and decision datasets in several visuomotor tasks, such as meal preparation (Li, Liu, and Rehg 2018), human-to-human social interactions (Zuo et al 2018), driving (Palazzi et al 2018), and Atari game playing (Zhang et al 2020b). So far these datasets have been used for modeling human attention.…”

“…We use human expert gaze data from Atari-HEAD dataset (Zhang et al 2020b) due to their semi-frame-byframe data collection protocol. The original game runs continually at 60Hz (Bellemare et al 2013), a speed that is challenging even for professional gamers.…”

“…We found that human eye movements were rushed and inaccurate at this speed hence could not serve as a useful reference in our comparison. In Zhang et al (2020b), however, the human data were collected in a semi-frame-by-frame mode, a design that allowed enough time for the players to attend to multiple relevant objects on the screen and make decisions. The slowed speed also allowed players to achieve world record level performance (Zhang et al 2020b).…”

“…Through these tools one can visualize an RL agent's saliency map given an image, which is a topographically arranged map that assigns an importance weight to each image pixel and is often treated as the "attention map" of the RL agent. On the other hand, a human attention dataset on Atari games has been collected using eye trackers and is made publicly available (Zhang et al 2020b), which makes it possible to do the comparison.…”