Operant conditioning is characterized by the contingent reinforcement of a designated behavior. Previously, feeding behavior in Aplysia has been demonstrated to be modified by operant conditioning, and a neural pathway (esophageal nerve; E n.) that mediates some aspects of reinforcement has been identified. As a first step toward a cellular analysis of operant conditioning, we developed an in vitro buccal ganglia preparation that expressed the essential features of operant conditioning. Motor patterns that represented at least two different aspects of fictive feeding (i.e., ingestion-like and rejection-like motor patterns) were elicited by tonic stimulation of a peripheral buccal nerve (n.2,3). Three groups of preparations were examined. In a contingent-reinforcement group, stimulation of E n. was contingent on the expression of a specific type of motor pattern (i.e., either ingestion-like or rejection-like). In a yoke-control group, stimulation of E n. was not contingent on any specific pattern. In a control group, E n. was not stimulated. The frequency of the reinforced pattern increased significantly only in the contingent-reinforcement group. No changes were observed in nonreinforced patterns or in the motor patterns of the control and yoke-control groups. Contingent reinforcement of the ingestion-like pattern was associated with an enhancement of activity in motor neuron B8, and this enhancement was specific to the reinforced pattern. These results suggest that the isolated buccal ganglia expressed an essential feature of operant conditioning (i.e., contingent reinforcement modified a designated operant) and that this analog of operant conditioning is accessible to cellular analysis. Key words: buccal ganglia; Aplysia californica; central pattern generator; operant conditioning; learning and memory; contingent reinforcementOperant conditioning, which was introduced by Thorndike (1911), is an example of associative learning in which an association is established between a specific behavior (the operant) and a stimulus (the reinforcement). A key feature of operant conditioning is the contingency of the reinforcement (i.e., the correlation between the expression of a designated operant behavior and the delivery of a reinforcement; Skinner, 1938;Konorski, 1948). As a result of this contingency the frequency of the reinforced behavior is modified. This phenomenon, known as the "law of effect" (Thorndike, 1933), provided evidence that the nervous system has mechanisms by which a particular motor output can be selected from among many different behaviors that may be expressed.Rhythmic motor acts such as locomotion, feeding, respiration, and heart rate can be modified by operant conditioning (Skinner, 1938;Miller, 1969;Cook and C arew, 1986;Susswein et al., 1986;Jaeger et al., 1987; L ukowiak et al., 1996). It is believed generally that rhythmic motor acts are mediated by groups of neurons referred to as central pattern generators (C PGs;Delcomyn, 1980;Selverston and Moulins, 1985). C PGs are multif unctional netwo...

Previously, an analog of operant conditioning was developed using the buccal ganglia of Aplysia, the probabilistic occurrences of a specific motor pattern (i.e., pattern I), a contingent reinforcement (i.e., stimulation of the esophageal nerve), and monotonic stimulation of a peripheral nerve (i.e., n.2,3). This analog expressed a key feature of operant conditioning (i.e., selective enhancement of the probability of occurrence of a designated motor pattern by contingent reinforcement). In addition, the training induced changes in the dynamical properties of neuron B51, an element of the buccal central pattern generator. To gain insights into the neuronal mechanisms that mediate features of operant conditioning, the present study identified a neuronal element that was critically involved in the selective enhancement of pattern I. We found that bursting activity in cell B51 contributed significantly to the expression of pattern I and that changes in the dynamical properties of this cell were associated with the selective enhancement of pattern I. These changes could be induced by an explicit association of reinforcement with random depolarization of B51. No stimulation of n.2,3 was required. These results indicate that the selection of a designated motor pattern by contingent reinforcement and the underlying neuronal plasticity resulted from the association of reinforcement with a component of central neuronal activity that contributes to a specific motor pattern. The sensory stimulus that allows for occurrences of different motor acts may not be critical for induction of plasticity that mediates the selection of a motor output by contingent reinforcement in operant conditioning.

Previously, an analog of operant conditioning in Aplysia was developed using the rhythmic motor activity in the isolated buccal ganglia. This analog expressed a key feature of operant conditioning, namely a selective enhancement in the occurrence of a designated motor pattern by contingent reinforcement. Different motor patterns generated by the buccal central pattern generator were induced by monotonic stimulation of a peripheral nerve (i.e., n.2,3). Phasic stimulation of the esophageal nerve (E n.) was used as an analog of reinforcement. The present study investigated the neuronal mechanisms associated with the genesis of different motor patterns and their modifications by contingent reinforcement. The genesis of different motor patterns was related to changes in the functional states of the pre-motor neuron B51. During rhythmic activity, B51 dynamically switched between inactive and active states. Bursting activity in B51 was associated with, and predicted, characteristic features of a specific motor pattern (i.e., pattern I). Contingent reinforcement of pattern I modified the dynamical properties of B51 by decreasing its resting conductance and threshold for eliciting plateau potentials and thus increased the occurrences of pattern I-related activity in B51. These modifications were not observed in preparations that received either noncontingent reinforcement (i.e., yoke control) or no reinforcement (i.e., control). These results suggest that a contingent reinforcement paradigm can regulate the dynamics of neuronal activity that is centrally programmed by the intrinsic cellular properties of neurons.

Motivated behaviors comprise appetitive actions whose occurrence results partly from an internally driven incentive to act. Such impulsive behavior can also be regulated by external rewarding stimuli that, through learning processes, can lead to accelerated and seemingly automatic, compulsive-like recurrences of the rewarded act. Here, we explored such behavioral plasticity in Aplysia by analyzing how appetitive reward stimulation in a form of operant conditioning can modify a goal-directed component of the animal's food-seeking behavior. In naive animals, protraction/retraction cycles of the tongue-like radula are expressed sporadically with highly variable interbite intervals. In contrast, animals that were previously given a food-reward stimulus in association with each spontaneous radula bite now expressed movement cycles with an elevated frequency and a stereotyped rhythmic organization. This rate increase and regularization, which was retained for several hours after training, depended on both the reward quality and its contingency because accelerated, stereotyped biting was not induced in animals that had previously received a less-palatable food stimulus or had been subjected to nonassociative reward stimulation. Neuronal correlates of these learning-induced changes were also expressed in the radula motor pattern-generating circuitry of isolated buccal ganglia. In such in vitro preparations, moreover, manipulation of the burst frequency of the bilateral motor pattern-initiating B63 interneurons indicated that the regularization of radula motor pattern generation in contingently trained animals occurred separately from an increase in cycle rate, thereby suggesting independent processes of network plasticity. These data therefore suggest that operant conditioning can induce compulsive-like actions in Aplysia feeding behavior and provide a substrate for a cellular analysis of the underlying mechanisms.

Our results show that plasticity in the spatiotemporal organization of pacemaker bursting, both within and between components of an action-initiating neuronal subcircuit, provides novel cellular substrates by which operant learning alters the recurrent expression of a simple goal-directed behavior.

Feeding behavior in Aplysia can be modified by operant conditioning in which contingent reinforcement is conveyed by the esophageal nerve (E n.). A neuronal analogue of this conditioning in the isolated buccal ganglia was developed by using stimulation of E n. as an analogue of contingent reinforcement. Previous studies indicated that E n. may release dopamine. We used a dopamine antagonist (methylergonovine) to investigate whether dopamine mediated the enhancement of motor patterns in the analogue of operant conditioning. Methylergonovine blocked synaptic connections from the reinforcement pathway and the contingent-dependent enhancement of the reinforced pattern. These results suggest that dopamine mediates at least part of the neuronal modifications induced by contingent reinforcement.

These data establish direct causality for the contributions of distinct synaptic and nonsynaptic adaptive processes to complementary facets of a compulsive behavior resulting from operant reward learning.

Whereas major insights into the neuronal basis of adaptive behavior have been gained from the study of automatic behaviors, including reflexive and rhythmic motor acts, the neural substrates for goal-directed behaviors in which decision-making about action selection and initiation are crucial, remain poorly understood. However, the mollusk Aplysia is proving to be increasingly relevant to redressing this issue. The functional properties of the central circuits that govern this animal’s goal-directed feeding behavior and particularly the neural processes underlying the selection and initiation of specific feeding actions are becoming understood. In addition to relying on the intrinsic operation of central networks, goal-directed behaviors depend on external sensory inputs that through associative learning are able to shape decision-making strategies. Here, we will review recent findings on the functional design of the central network that generates Aplysia’s feeding-related movements and the sensory-derived plasticity that through learning can modify the selection and initiation of appropriate action. The animal’s feeding behavior and the implications of decision-making will be briefly described. The functional design of the underlying buccal network will then be used to illustrate how cellular diversity and the coordination of neuronal burst activity provide substrates for decision-making. The contribution of specific synaptic and neuronal membrane properties within the buccal circuit will also be discussed in terms of their role in motor pattern selection and initiation. The ability of learning to “rigidify” these synaptic and cellular properties so as to regularize network operation and lead to the expression of stereotyped rhythmic behavior will then be described. Finally, these aspects will be drawn into a conceptual framework of how Aplysia’s goal-directed circuitry compares to the central pattern generating networks for invertebrate rhythmic behaviors.