A computer simulation/mathematical model of learning: Extension of DMOD from appetitive to aversive situations

Abstract

The computer simulation/mathematical model called DMOD, which can simulate over 35 different phenomena in appetitive discrete-trial and simple free-operant situations, has been extended to include aversive di~rete-trial situations. Learning (V) is calculated using a three-parameter equation .1V = a{j(A -V) (see Daly & Daly, 1982;Rescorla & Wagner, 1972). The equation is applied to three possible goal events in the appetitive (e.g., food)case and to three in the aversive (e.g., shock) case. The original goal event can be present, absent, or reintroduced; in the appetitive situation, these events condition approach (Vap), avoidance (Vav), and courage (Vee), respectively. In the aversive situation, the events condition avoidance (Vav*), approach (Vap*), and cowardice (Vcc*), respectively. The model was developed in simple learning situations and subsequently was applied to complex situations. It can account for such diverse phenomena as contrast effects after reward shifts, greater persistence following partial than following continuous reinforcement, and a preference for predictable appetitive and predictable aversive events. Application of the aversive version of the model to "reward" shifts is described.Our goal is to develop a computer simulation/mathematical model of learning that is as simple as possible with as much breadth as possible. The model, called DMOD (Daly MODification of the Rescorla-Wagner Model), is simple because learning is calculated with one simple equation using three parameters. It is applied to diverse situations by assuming that there are a number of different goal events possible, each of which conditions either approach or avoidance of the goal. It was originally developed to account for appetitive, discrete-trial experiments, and can currently account for behavior in over 30 different paradigms (see Daly & Daly, 1982). DMOD was then extended to simulate the effect of simple schedules of reinforcement in free-operant experiments (Daly & Daly, 1984b). Our purpose is to outline the extension of DMOD to the aversive case and to show its application in a complicated experimental situation. To understand the rationale behind the extension, however, it is neces- sary to review development of the model in the appetitive case and the rules we follow for development of DMOD. SELECTION OF PHENOMENA TO BE SIMULATEDWe believe that the initial goal of a new model is to be able to account for well-established and replicable phenomena. We feel that it is dangerous to develop a model around recently discovered phenomena, because the boundary conditions under which they can be obtained and the variables influencing them are unknown. The purpose of a theory, however, is not only to integrate existing replicable data, but also to correctly predict new results. The primary phenomenon we use to test predictions of the appetitive version of DMOD is acquisition of a preference for predictable reward. The model made some interesting predictions concerning when a preference for the unpredictable reward situati...

Changes in learning about aversive nonreward accounts for ontogeny of paradoxical appetitive reward effects in the rat pup: A mathematical model (DMOD) integrates results.

“…DMOD has been used successfully to integrate over 60 appetitive and aversive learning phenomena (Daly & Daly, 1982, 1987, 1991). The model has also been used to predict behaviors in areas as diverse as (a) preference for unpredictable over predictable reward (Daly, 1985, 1986, 1989, in press) and (b) the behavioral changes in laboratory rats after they consumed salmon from Lake Ontario, which are environmentally contaminated with toxic chemicals (Daly, Hertzler, & Sargent, 1989).…”

Persistence and the importance of nonreward: Some applications of frustration theory and DMOD

“…In 1987, we published an extension of DMOD that applies to aversive conditioning (e.g., electric shocks). Simulations of over 60 different learning phenomena have now been published (see Daly, 1985Daly, , 1989Daly, , 1991aDaly, , 1992bDaly & Daly, 1987.…”

“…To condition T{;c, the lambda value is the discrepancy between the amount of frustration experienced (0 on a rewarded trial) and the amount of frustration expected (indexed by V.v) ' (See Daly & Daly, 1982, 1987 On the basis of the research results ofothers, three additional assumptions are made: (1) The importance of a negative event is greater than that ofa positive event (the lambda value for Vav is twice the discrepancy; see Daly & Daly, 1982, p. 446). (2) Negative but not positive V values show spatial generalization (we multiply Vav by .5 to determine the start speed measure assumed to be about 180 cm from the goal, and.…”

Preference for unpredictable food rewards occurs with high proportion of reinforced trials or alcohol if rewards are not delayed.

“…Only recently have studies been done to uncover the combination of conditions that result in a preference for the unpredictable reward situation: Daly (1985) showed that a small 1-pellet reward in the immediate reward condition resulted in a 75% preference for the unpredictable reward situation, whereas a large 15-pellet reward resulted in a 70% preference for the predictable reward situation. Only one theory predicts that subjects will prefer the unpredictable reward situation if rewards are small and not delayed: DMOD (Daly MODification of the Rescorla & Wagner, 1972, model; Daly & Daly, 1982, 1984, 1987). Computer simulations of this model showed that there is a preference for unpredictability due, in part, to blocking, which is reversed only when unpredicted nonreward is aversive.…”