Significant trial-by-trial variation persists even in the most practiced skills. One prevalent view is that such variation is simply 'noise' that the nervous system is unable to control or that remains below threshold for behavioural relevance. An alternative hypothesis is that such variation enables trial-and-error learning, in which the motor system generates variation and differentially retains behaviours that give rise to better outcomes. Here we test the latter possibility for adult bengalese finch song. Adult birdsong is a complex, learned motor skill that is produced in a highly stereotyped fashion from one rendition to the next. Nevertheless, there is subtle trial-by-trial variation even in stable, 'crystallized' adult song. We used a computerized system to monitor small natural variations in the pitch of targeted song elements and deliver real-time auditory disruption to a subset of those variations. Birds rapidly shifted the pitch of their vocalizations in an adaptive fashion to avoid disruption. These vocal changes were precisely restricted to the targeted features of song. Hence, birds were able to learn effectively by associating small variations in their vocal behaviour with differential outcomes. Such a process could help to maintain stable, learned song despite changes to the vocal control system arising from ageing or injury. More generally, our results suggest that residual variability in well learned skills is not entirely noise but rather reflects meaningful motor exploration that can support continuous learning and optimization of performance.

In songbirds, the basal ganglia outflow nucleus LMAN is a cortical analog that is required for several forms of song plasticity and learning. Moreover, in adults, inactivating LMAN can reverse the initial expression of learning driven via aversive reinforcement. In the present study, we investigated how LMAN contributes to both reinforcement-driven learning and a self-driven recovery process in adult Bengalese finches. We first drove changes in the fundamental frequency of targeted song syllables and compared the effects of inactivating LMAN with the effects of interfering with N-methyl-d-aspartate (NMDA) receptor-dependent transmission from LMAN to one of its principal targets, the song premotor nucleus RA. Inactivating LMAN and blocking NMDA receptors in RA caused indistinguishable reversions in the expression of learning, indicating that LMAN contributes to learning through NMDA receptor-mediated glutamatergic transmission to RA. We next assessed how LMAN's role evolves over time by maintaining learned changes to song while periodically inactivating LMAN. The expression of learning consolidated to become LMAN independent over multiple days, indicating that this form of consolidation is not completed over one night, as previously suggested, and instead may occur gradually during singing. Subsequent cessation of reinforcement was followed by a gradual self-driven recovery of original song structure, indicating that consolidation does not correspond with the lasting retention of changes to song. Finally, for self-driven recovery, as for reinforcement-driven learning, LMAN was required for the expression of initial, but not later, changes to song. Our results indicate that NMDA receptor-dependent transmission from LMAN to RA plays an essential role in the initial expression of two distinct forms of vocal learning and that this role gradually wanes over a multiday process of consolidation. The results support an emerging view that cortical-basal ganglia circuits can direct the initial expression of learning via top-down influences on primary motor circuitry.

Reinforcement signals indicating success or failure are known to alter the probability of selecting between distinct actions. However, successful performance of many motor skills, such as speech articulation, also requires learning behavioral trajectories that vary continuously over time. Here, we investigated how temporally discrete reinforcement signals shape a continuous behavioral trajectory, the fundamental frequency of adult Bengalese finch song. We provided reinforcement contingent on fundamental frequency performance only at one point in song. Learned changes to fundamental frequency were maximal at this point, but also extended both earlier and later in the fundamental frequency trajectory. A simple principle predicted the detailed structure of learning; birds learn to produce the average of the behavioral trajectories associated with successful outcomes. This learning rule accurately predicts the structure of learning at a millisecond time scale, demonstrating that the nervous system records fine-grained details of successful behavior and uses this information to guide learning.

Variation in sequencing of actions occurs in many natural behaviors, yet how such variation is maintained is poorly understood. We investigated maintenance of sequence variation in adult Bengalese finch song, a learned skill with rendition-to-rendition variation in the sequencing of discrete syllables (i.e., syllable "b" might transition to "c" with 70% probability and to "d" with 30% probability). We found that probabilities of transitions ordinarily remain stable but could be modified by delivering aversive noise bursts following one transition (e.g., "b3c") but not the alternative (e.g., "b3d"). Such differential reinforcement induced gradual, adaptive decreases in probabilities of targeted transitions and compensatory increases in alternative transitions. Thus, the normal stability of transition probabilities does not reflect hardwired premotor circuitry. While all variable transitions could be modified by differential reinforcement, some were less readily modified than others; these were cases that exhibited more alternation between possible transitions than predicted by chance (i.e., "b3d " would tend to follow "b3c " and vice versa). These history-dependent transitions were less modifiable than more stochastic transitions. Similarly, highly stereotyped transitions (which are completely predictable) were not modifiable. This suggests that stochastically generated variability is crucial for sequence modification. Finally, we found that, when reinforcement ceased, birds gradually restored transition probabilities to their baseline values. Hence, the nervous system retains a representation of baseline probabilities and has the impetus to restore them. Together, our results indicate that variable sequencing in a motor skill can reflect an end point of learning that is stably maintained via continual self-monitoring.