In associative learning in mammals, it is widely accepted that the discrepancy, or error, between actual and predicted reward determines whether learning occurs. Complete evidence for the prediction error theory, however, has not been obtained in any learning systems: Prediction error theory stems from the finding of a blocking phenomenon, but blocking can also be accounted for by other theories, such as the attentional theory. We demonstrated blocking in classical conditioning in crickets and obtained evidence to reject the attentional theory. To obtain further evidence supporting the prediction error theory and rejecting alternative theories, we constructed a neural model to match the prediction error theory, by modifying our previous model of learning in crickets, and we tested a prediction from the model: the model predicts that pharmacological intervention of octopaminergic transmission during appetitive conditioning impairs learning but not formation of reward prediction itself, and it thus predicts no learning in subsequent training. We observed such an “auto-blocking”, which could be accounted for by the prediction error theory but not by other competitive theories to account for blocking. This study unambiguously demonstrates validity of the prediction error theory in associative learning.

Revealing reinforcing mechanisms in associative learning is important for elucidation of brain mechanisms of behavior. In mammals, dopamine neurons are thought to mediate both appetitive and aversive reinforcement signals. Studies using transgenic fruit-flies suggested that dopamine neurons mediate both appetitive and aversive reinforcements, through the Dop1 dopamine receptor, but our studies using octopamine and dopamine receptor antagonists and using Dop1 knockout crickets suggested that octopamine neurons mediate appetitive reinforcement and dopamine neurons mediate aversive reinforcement in associative learning in crickets. To fully resolve this issue, we examined the effects of silencing of expression of genes that code the OA1 octopamine receptor and Dop1 and Dop2 dopamine receptors by RNAi in crickets. OA1-silenced crickets exhibited impairment in appetitive learning with water but not in aversive learning with sodium chloride solution, while Dop1-silenced crickets exhibited impairment in aversive learning but not in appetitive learning. Dop2-silenced crickets showed normal scores in both appetitive learning and aversive learning. The results indicate that octopamine neurons mediate appetitive reinforcement via OA1 and that dopamine neurons mediate aversive reinforcement via Dop1 in crickets, providing decisive evidence that neurotransmitters and receptors that mediate appetitive reinforcement indeed differ among different species of insects.

The effect of repetitive training on learned actions has been a major subject in behavioural neuroscience. Many studies of instrumental conditioning in mammals, including humans, suggested that learned actions early in training are goal-driven and controlled by outcome expectancy, but they become more automatic and insensitive to reduction in the value of the outcome after extended training. It was unknown, however, whether the development of value-insensitive behaviour also occurs by extended training of Pavlovian conditioning in any animals. Here we show that crickets Gryllus bimaculatus that had received minimal training to associate an odour with water (unconditioned stimulus, US) did not exhibit conditioned response (CR) to the odour when they were given water until satiation before the test, but those that had received extended training exhibited CR even when they were satiated with water. Further pharmacological experiments suggested that octopamine neurons, the invertebrate counterparts of noradrenaline neurons, mediate US value signals and control execution of CR after minimal training, but the control diminishes with the progress of training and hence the CR becomes insensitive to US devaluation. The results suggest that repetitive sensory experiences can lead to a change from a goal-driven response to a more automatic one in crickets.

In associative learning in mammals, it is widely accepted that the discrepancy, or error, between actual and predicted reward determines whether learning occurs. The prediction error theory has been proposed to account for the finding of a blocking phenomenon, in which pairing of a stimulus X with an unconditioned stimulus (US) could block subsequent association of a second stimulus Y to the US when the two stimuli were paired in compound with the same US. Evidence for this theory, however, has been imperfect since blocking can also be accounted for by competitive theories. We recently reported blocking in classical conditioning of an odor with water reward in crickets. We also reported an “auto-blocking” phenomenon in appetitive learning, which supported the prediction error theory and rejected alternative theories. The presence of auto-blocking also suggested that octopamine neurons mediate reward prediction error signals. Here we show that blocking and auto-blocking occur in aversive learning to associate an odor with salt water (US) in crickets, and our results suggest that dopamine neurons mediate aversive prediction error signals. We conclude that the prediction error theory is applicable to both appetitive learning and aversive learning in insects.

Ca2+/calmodulin (CaM)-dependent protein kinase II (CaMKII) is a key molecule in many systems of learning and memory in vertebrates, but roles of CaMKII in invertebrates have not been characterized in detail. We have suggested that serial activation of NO/cGMP signaling, cyclic nucleotide-gated channel, Ca2+/CaM and cAMP signaling participates in long-term memory (LTM) formation in olfactory conditioning in crickets, and here we show participation of CaMKII in LTM formation and propose its site of action in the biochemical cascades. Crickets subjected to 3-trial conditioning to associate an odor with reward exhibited memory that lasts for a few days, which is characterized as protein synthesis-dependent LTM. In contrast, animals subjected to 1-trial conditioning exhibited memory that lasts for only several hours (mid-term memory, MTM). Injection of a CaMKII inhibitor prior to 3-trial conditioning impaired 1-day memory retention but not 1-hour memory retention, suggesting that CaMKII participates in LTM formation but not in MTM formation. Animals injected with a cGMP analogue, calcium ionophore or cAMP analogue prior to 1-trial conditioning exhibited 1-day retention, and co-injection of a CaMKII inhibitor impaired induction of LTM by the cGMP analogue or that by the calcium ionophore but not that by the cAMP analogue, suggesting that CaMKII is downstream of cGMP production and Ca2+ influx and upstream of cAMP production in biochemical cascades for LTM formation. Animals injected with an adenylyl cyclase (AC) activator prior to 1-trial conditioning exhibited 1-day retention. Interestingly, a CaMKII inhibitor impaired LTM induction by the AC activator, although AC is expected to be a downstream target of CaMKII. The results suggest that CaMKII interacts with AC to facilitate cAMP production for LTM formation. We propose that CaMKII serves as a key molecule for interplay between Ca2+ signaling and cAMP signaling for LTM formation, a new role of CaMKII in learning and memory.