Characterizing the EEG Correlates of Exploratory Behavior

Abstract

Abstract-This study aims to characterize the EEG correlates of exploratory behavior. Decision making in an uncertain environment raises a conflict between two opposing needs: gathering information about the environment and exploiting this knowledge in order to optimize the decision. Exploratory behavior has already been studied using fMRI. Based on a usual paradigm in reinforcement learning, this study has shown bilateral activation in the frontal and parietal cortex. To our knowledge, no previous study has been done on it using EEG.The study of the exploratory behavior using EEG signals raises two difficulties. First, the labels of trial as exploitation or exploration cannot be directly derived from the subject action. In order to access this information, a model of how the subject makes his decision must be built. The exploration related information can be then derived from it. Second, because of the complexity of the task, its EEG correlates are not necessarily time locked with the action. So the EEG processing methods used should be designed in order to handle signals that shift in time across trials.Using the same experimental protocol as the fMRI study, results show that the bilateral frontal and parietal areas are also the most discriminant. This strongly suggests that the EEG signal also conveys information about the exploratory behavior.

The timing of exploratory decision-making revealed by single-trial topographic EEGanalyses

“…The validity of the behavioural model employed here has already been demonstrated (Bourdaud et al, 2008) by comparing its results with the actual statistical parameters of the machines.…”

“…Here, we use the same behavioral model proposed in Daw et al 2006 andBourdaud et al 2008, involving two steps for every trial: First, it provides an estimation of the payoff that users expect to receive from each machine and in a second step it estimates which machine subjects are supposed to choose. In the latter step each subject is considered separately in order to account for inter subject variability.…”

“…However, when they choose the machine that did not correspond to the one with the highest estimated payoff the decision is labeled as exploration, but only if this reward differs from the highest one by a threshold value (set to 4). Only exploratory trials corresponding to a machine change and exploitatory corresponding to a machine stay are kept for further analyses, in a similar way as was applied in another EEG study using the same task and subjects (Bourdaud et al, 2008). We used here exactly the same behavioral model in order to obtain comparable results.…”

“…In a previous study using the same subjects and paradigm (Bourdaud et al, 2008) it was shown that it is also possible to find discriminating oscillatory patterns of EEG activity of the current decision within 1000ms prior to button press. Consistently, we found an overlap between the time period that was used there with the time period that is used in the present study.…”

“…More generally, the prefrontal cortex and the anterior cingulate cortex have been repeatedly reported to be involved in decision-making under uncertainty (Hsu et al, 2005;Yoshida and Ishii, 2006;Hampton and O'Doherty 2007;Rushworth and Behrens 2008;Seo et al, 2009). Discrimination between exploratory/exploitatory decisions has also been documented using alpha and beta band EEG activity (Bourdaud et al, 2008).…”