Progress in Modeling Through Distributed Collaboration

Wills, Andy J.; O’Connell, Garret; Edmunds, Charlotte; Inkster, Angus

doi:10.1016/bs.plm.2016.11.007

Cited by 8 publications

(6 citation statements)

References 55 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Nevertheless, further investigation of both approaches, across a range of phenomena, would be a fruitful direction for future research. The importance of making broad relative adequacy comparisons of models has been previously emphasised within the literature (Wills & Pothos, 2012;Wills et al, 2017). Further research could also investigate a role for context in explaining the data (e.g.…”

Section: Other Accountsmentioning

confidence: 99%

Representing uncertainty in the Rescorla-Wagner model: Blocking, the redundancy effect, and outcome base rate

Spicer

Wills

Jones

et al. 2021

OJEPN

Self Cite

View full text Add to dashboard Cite

It is generally assumed that the Rescorla and Wagner (1972) model adequately accommodates the full results of simple cue competition experiments in humans (e.g. Dickinson et al., 1984), while the Bush and Mosteller (1951) model cannot. We present simulations that demonstrate this assumption is wrong in at least some circumstances. The Rescorla-Wagner model, as usually applied, fits the full results of a simple forward cue-competition experiment no better than the Bush-Mosteller model. Additionally, we present a novel finding, where letting the associative strength of all cues start at an intermediate value (rather than zero), allows this modified model to provide a better account of the experimental data than the (equivalently modified) Bush-Mosteller model. This modification also allows the Rescorla-Wagner model to account for a redundancy effect experiment (Uengoer et al., 2013); something that the unmodified model is not able to do. Furthermore, the modified Rescorla-Wagner model can accommodate the effect of varying the proportion of trials on which the outcome occurs (i.e. the base rate) on the redundancy effect (Jones et al., 2019). Interestingly, the initial associative strength of cues varies in line with the outcome base rate. We propose that this modification provides a simple way of mathematically representing uncertainty about the causal status of novel cues within the confines of the Rescorla-Wagner model. The theoretical implications of this modification are discussed. We also briefly introduce free and open resources to support formal modelling in associative learning. Keywords: associative learning, prediction error, uncertainty, modelling, blocking, redundancy effect, open science.

show abstract

Section: Other Accountsmentioning

confidence: 99%

Representing uncertainty in the Rescorla-Wagner model: Blocking, the redundancy effect, and outcome base rate

Spicer

Wills

Jones

et al. 2021

OJEPN

Self Cite

View full text Add to dashboard Cite

show abstract

“…Further progress requires us to understand the mechanism by which goal-directed and habitual systems assign value to actions. Here, computational models of learning can be usefully brought to bear insofar as they signpost optimal information processing (to which natural selection tends, if not unerringly), express key computational terms for which we may hope to find behavioural or neural correlates, and express this information in unequivocal terms (Wills, et al 2017;Wills and Pothos 2012). The properties of goal-directed and habit led behaviour strongly suggest they are underlain by two distinct forms of learning, termed model-based and model-free (Daw and O' Doherty 2013).…”

Section: Introductionmentioning

confidence: 99%

Model-free and model-based reward prediction errors in EEG

et al. 2018

Self Cite

View full text Add to dashboard Cite

Learning theorists posit two reinforcement learning systems: model-free and model-based. Model-based learning incorporates knowledge about structure and contingencies in the world to assign candidate actions with an expected value. Model-free learning is ignorant of the world's structure; instead, actions hold a value based on prior reinforcement, with this value updated by expectancy violation in the form of a reward prediction error. Because they use such different learning mechanisms, it has been previously assumed that model-based and model-free learning are computationally dissociated in the brain. However, recent fMRI evidence suggests that the brain may compute reward prediction errors to both model-free and model-based estimates of value, signalling the possibility that these systems interact. Because of its poor temporal resolution, fMRI risks confounding reward prediction errors with other feedback-related neural activity. In the present study, EEG was used to show the presence of both model-based and model-free reward prediction errors and their place in a temporal sequence of events including state prediction errors and action value updates. This demonstration of model-based prediction errors questions a long-held assumption that model-free and model-based learning are dissociated in the brain.

show abstract

“…We chose the IBRE for two reasons. First, empirically, it is a well-replicated phenomenon, and so an appropriate target for modeling (see Wills, O'Connell, Edmunds, & Inkster, 2017, for a further discussion of this point). Second, theoretically, it is non-trivial to explain, with only relatively few formal models of learning able to capture the main group-level results.…”

Section: Applying G-distance To the Inverse Base-rate Effectmentioning

confidence: 99%

g-distance: On the comparison of model and human heterogeneity

Dome¹,

Wills²

2023

Preprint

View full text Add to dashboard Cite

Models are often evaluated when their behavior is at its closest to a single group-averaged empirical result, but this evaluation neglects the fact that both model and human behavior can be heterogeneous. Here we develop a measure, g-distance, which considers model adequacy as the extent to which models exhibit a similar range of behaviors to the humans they model. We demonstrate an application of this measure to five models of an irrational learning effect, the inverse base-rate effect. None of the five models we evaluate is substantially better than a random model on our metric. In the process of analyzing the human data, we also discovered that the group-level result was observed in less than 1% of individuals. We discuss the implications of our findings for model evaluation generally, and for models of the inverse base-rate effect specifically. We end by suggesting future avenues of research in computational modeling.

show abstract

Progress in Modeling Through Distributed Collaboration

Cited by 8 publications

References 55 publications

Representing uncertainty in the Rescorla-Wagner model: Blocking, the redundancy effect, and outcome base rate

Representing uncertainty in the Rescorla-Wagner model: Blocking, the redundancy effect, and outcome base rate

Model-free and model-based reward prediction errors in EEG

g-distance: On the comparison of model and human heterogeneity

Contact Info

Product

Resources

About