Punishing an Error Improves Learning: The Influence of Punishment Magnitude on Error-Related Neural Activity and Subsequent Learning

Abstract

Punishing an error to shape subsequent performance is a major tenet of individual and societal level behavioral interventions. Recent work examining error-related neural activity has identified that the magnitude of activity in the posterior medial frontal cortex (pMFC) is predictive of learning from an error, whereby greater activity in this region predicts adaptive changes in future cognitive performance. It remains unclear how punishment influences error-related neural mechanisms to effect behavior change, particularly in key regions such as pMFC, which previous work has demonstrated to be insensitive to punishment. Using an associative learning task that provided monetary reward and punishment for recall performance, we observed that when recall errors were categorized by subsequent performance-whether the failure to accurately recall a number-location association was corrected at the next presentation of the same trial-the magnitude of error-related pMFC activity predicted future correction. However, the pMFC region was insensitive to the magnitude of punishment an error received and it was the left insula cortex that predicted learning from the most aversive outcomes. These findings add further evidence to the hypothesis that error-related pMFC activity may reflect more than a prediction error in representing the value of an outcome. The novel role identified here for the insular cortex in learning from punishment appears particularly compelling for our understanding of psychiatric and neurologic conditions that feature both insular cortex dysfunction and a diminished capacity for learning from negative feedback or punishment.

Behavioral and Neural Properties of Social Reinforcement Learning

“…In addition, participants showed faster reaction times after trials that did not provide positive social reinforcement. This finding is similar to studies that demonstrate improved performance on a trial that follows receiving punishment (Hester et al, 2010) or choosing to make a bet more often after losing money than winning money (Liu et al, 2007), though the present study did not assess strategic behavior directly. Taken together, the behavioral findings demonstrate that participants learned the reinforcement contingencies and thus provide an objective index of social learning.…”

Improving Robot Motor Learning with Negatively Valenced Reinforcement Signals

“…These results could be considered contradictory to other studies (Hester et al, 2010), where punishment-driven learning has been found to improve learning performance; however, the nature of the tested tasks is different, i.e., procedural learning vs. associative learning, respectively. Besides, the recognized existence of differential pathways for reward- and punishment-driven learning reconciles both results.…”

Spatio-temporal Brain Dynamics Mediating Post-error Behavioral Adjustments

“…Interestingly, recent evidence shows that modifications in learned stimulusresponse mapping associations depend on error-related activity within medial pFCs when feedback is provided to participants (Hester et al, 2010). This finding indicates that processes related to monitoring and reweighting of a behaviorʼs value parallel those related to increases in executive control within prefrontal areas following errors; both mechanisms impacting how subsequent stimuli are handled.…”

“…Their results suggest a role for prefrontal areas, notably the right ventrolateral prefrontal area, in PES. Reinforcement learning studies have also pointed out associations between activity in prefrontal areas and associative learning (Brown & Braver, 2005;Nieuwenhuis, Holroyd, Mol, & Coles, 2004;Holroyd & Coles, 2002), and more specifically, this literature reported associations between errorrelated activity in pFCs and immediate changes in post-error behavior (Hester, Murphy, Brown, & Skilleter, 2010;Hester, Barre, Murphy, Silk, & Mattingley, 2008;Frank, Woroch, & Curran, 2005;Frank, Seeberger, & OʼReilly, 2004). However, the low temporal resolution of fMRI technique used in their study did not allow for disentangling the precise dynamics of brain mechanisms underlying post-error behavioral adjustments.…”