Pupil Dilation and Response Slowing Distinguish Deliberate Explorative Choices in the Probabilistic Learning Task

Abstract

This study examined whether pupil size and response time would distinguish directed exploration from random exploration and exploitation. Eighty-nine participants performed the two-choice probabilistic learning task while their pupil size and response time were continuously recorded. Using LMM analysis, we estimated differences in the pupil size and response time between the advantageous and disadvantageous choices as a function of learning success, i.e., whether or not a participant has learned the probabilistic contingency between choices and their outcomes. We proposed that before a true value of each choice became known to a decision-maker, both advantageous and disadvantageous choices represented a random exploration of the two options with an equally uncertain outcome, whereas the same choices after learning manifested exploitation and direct exploration strategies, respectively. We found that disadvantageous choices were associated with increases both in response time and pupil size, but only after the participants had learned the choice-reward contingencies. For the pupil size, this effect was strongly amplified for those disadvantageous choices that immediately followed gains as compared to losses in the preceding choice. Pupil size modulations were evident during the behavioral choice rather than during the pretrial baseline. These findings suggest that occasional disadvantageous choices, which violate the acquired internal utility model, represent directed exploration. This exploratory strategy shifts choice priorities in favor of information seeking and its autonomic and behavioral concomitants are mainly driven by the conflict between the behavioral plan of the intended exploratory choice and its strong alternative, which has already proven to be more rewarding.

Losses resulting from deliberate exploration trigger beta oscillations in frontal cortex

“…Crucially, the magnitude of a large-scale β suppression, when a subject formed his/her decision reliably, predicted decision costs (with response time taken as a measurable proxy of this internal variable) on a single trial basis. Given multiple evidence associating the strength of β suppression with greater cognitive and attentional efforts 16–18 , this finding strongly suggests that such a choice requires additional resources to overcome the internal utility model favoring the advantageous alternative and strongly supports the hypothesis of its deliberately explorative nature 13,33 .…”

“…This conflict arises between at least two simultaneously active competing internal models, or ‘task sets’ 9,12 – one being a predominant response tendency (exploitation), and the other – its conscious alternative (exploration). Our recent pupillometric study lends support to this assumption 13 : we found that such explorative choices compared to exploitative ones are accompanied by larger pupil dilation and longer decision time. We speculated that this state of conflict supposedly entails an increase in the degree of processing required to make the deliberately explorative decisions.…”

“…We addressed the influence of the previous feedback because it affects behavioral and pupillometric measures on the next trial 13…”

“…A recent pupillometric study 13 revealed that advantageous choices that immediately preceded and immediately followed exploratory choices significantly differed from the advantageous choices committed within the periods of continuous exploitation. Thus, we used the following four levels of Choice type factor (Fig.…”

“…Finally, we aimed to evaluate how the immediate history of punishments and rewards in the previous trial affected the brain response to the feedback on the current trial. We addressed the influence of the previous feedback because it affects behavioral and pupillometric measures on the next trial 13 . An additional factor Previous feedback corresponding to the outcome of the previous trial (two levels: punishment and reward) was included into the LMM model: …”