HeiDI: A model for Pavlovian learning and performance with reciprocal associations.

Abstract

Associative treatments of how Pavlovian conditioning affects conditioned behavior are rudimentary: A simple ordinal mapping is held to exist between the strength of an association (V) between a conditioned stimulus (CS) and an unconditioned stimulus (US;i.e., VCS-US) and conditioned behavior in a given experimental preparation. The inadequacy of this simplification is highlighted by recent studies that have taken multiple measures of conditioned behavior: Different measures of conditioned behavior provide the basis for drawing opposite conclusions about VCS-US across individual animals. Here, we develop a simple model involving reciprocal associations between the CS and US (VCS-US and VUS-CS) that simulates these qualitative individual differences in conditioned behavior. The new model, HeiDI (How excitation and inhibition Determine Ideo-motion), enables a broad range of phenomena to be accommodated, which are either beyond the scope of extant models or require them to appeal to additional (learning) processes. It also provides an impetus for new lines of inquiry and generates novel predictions. c 1 c 1 6. Brimer, C.J. (1970). Inhibition and disinhibition of an operant response as a function of the amount and type of prior training.

Higher-Order Conditioning: What Is Learnt and How it Is Expressed

“…The formation of reciprocal associations between stimulus 1 and stimulus 2, having perceived intensities of α 1 and α 2 , is determined by two equations: ∆V 1-2 = α 1 (c.α 2 − V TOTAL-2 ); and ∆V 2-1 = α 2 (c.α 1 − V TOTAL-1 ) 2 . These rules underpin the HeiDI model (Honey et al, 2020a). For both equations, associative changes on a given trial (∆V 1-2 and ∆V 2-1 ) are influenced by pooled error terms (i.e., c.α 2 − V TOTAL-2 and c.α 1 − V TOTAL-1 ) in which V TOTAL-2 and V TOTAL-1 are the summed associative strengths of stimuli present on that trial to the subscripted stimulus ( 1 or 2 ).…”

“…The issue then becomes one of specifying how the combined associative strengths within the extended associative structures (see Figure 3) is distributed to reflect the properties of A through the response units it is connected to and those of the retrieved representations of X and US. Following HeiDI (Honey et al, 2020a), we assume that they do so in proportion to their perceived intensities: for example, if the perceived intensity of A is higher than that of the retrieved memories of X or the US then a greater proportion of the combined associative strength would generate responses that are linked to A. Finally, we assume that this process is modulated by the similarity between the perceived intensities of the stimuli presented at the test (e.g., the associatively retrieved memory of X) to their perceived intensities on the conditioning trials (see Hall, 1996, 1998;Ward-Robinson et al, 2001;Honey, 2011, 2016;Lin et al, 2013; see also, Kamil, 1969; see also, Cole et al, 1995;Barnet and Miller, 1996).…”

“…Otherwise, the learning rules are simplified extensions to the one developed by Rescorla and Wagner (1972;see also, McLaren et al, 1989) 4 . Equations 1 and 2 reference these generic equations to the critical A→X and X→A associations, and Equations 3 and 4 reference them to the X→US and 2 The constant (c = 1 in units of V) is required to balance the equations in terms of the dimensions/units involved (see Honey et al, 2020a). 3 The fact that the asymptotes and the rates at which they are reached are determined by α 1 and α 2 creates computational advantages when specifying the similarity of (1) the retrieved values of α 2 and α 1 (given by the numerical values of V 1-2 and V 2-1 , respectively), and (2) their conditioned values α 2 and α 1 .…”

“…The influence of R A , R X, and R US on the response-generating units (r1-r6 in Figure 3) will reflect the strengths of the unconditioned links between A, X and the US and r1-r6; for example, through multiplying R A , R X, and R US by the weights from A, X and the US to r1-r6 (see Honey et al, 2020a). Figure 4 presents some indicative simulations of the values of R A , R X, and R US .…”

“…In fact, a recent critical review of evidence relating to these accounts suggested that they leave many important issues unresolved, which motivated the development of a new computational model of higher-order conditioning (Honey and Dwyer, under review). This model was built on a recent analysis of Pavlovian conditioning and performance: HeiDI (Honey et al, 2020a). Here, we first present a synthesis of extant informal accounts of higher-order conditioning together with the evidence that they fail to address, before presenting the new computational model of higher-order conditioning.…”