Through the looking glass: a dynamic lens model approach to multiple cue probability learning

Shanks, David R.; Speekenbrink, Maarten

doi:10.1093/acprof:oso/9780199216093.003.0018

Cited by 5 publications

(3 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Another related paradigm is multiple cue probability learning (MCPL; Kruschke & Johansen, 1999; Speekenbrink & Shanks, 2008) in which participants are shown an array of cues that are probabilistically related to an outcome and have to learn the underlying function mapping the cues’ features to expected outcomes. Especially when the outcome is a categorical variable, such as in the well-known weather prediction task (Gluck, Shohamy, & Myers, 2002; Speekenbrink, Channon, & Shanks, 2008), making a prediction is structurally similar to a decision between multiple arms (possible predictions) that are rewarded (correct prediction) or not (incorrect prediction).…”

Section: Cmabsmentioning

confidence: 99%

Putting bandits into context: How function learning supports decision making.

Schulz¹,

Konstantinidis

Speekenbrink³

2018

Journal of Experimental Psychology: Learning, Memory, and Cogni

Self Cite

View full text Add to dashboard Cite

We introduce the contextual multi-armed bandit task as a framework to investigate learning and decision making in uncertain environments. In this novel paradigm, participants repeatedly choose between multiple options in order to maximise their rewards. The options are described by a number of contextual features which are predictive of the rewards through initially unknown functions. From their experience with choosing options and observing the consequences of their decisions, participants can learn about the functional relation between contexts and rewards and improve their decision strategy over time. In three experiments, we explore participants' behaviour in such learning environments. We predict participants' behaviour by context-blind (mean-tracking, Kalman filter) and contextual (Gaussian process and linear regression) learning approaches combined with different choice strategies. Participants are mostly able to learn about the context-reward functions and their behaviour is best described by a Gaussian process learning strategy which generalizes previous experience to similar instances. In a relatively simple task with binary features, they seem to combine this learning with a "probability of improvement" decision strategy which focuses on alternatives that are expected to lead to an improvement upon a current favourite option. In a task with continuous features that are linearly related to the rewards, participants seem to more explicitly balance exploration and exploitation. Finally, in a difficult learning environment where the relation between features and rewards is non-linear, some participants are again well-described by a Gaussian process learning strategy, whereas others revert to context-blind strategies.

show abstract

Section: Cmabsmentioning

confidence: 99%

Putting bandits into context: How function learning supports decision making.

Schulz¹,

Konstantinidis

Speekenbrink³

2018

Journal of Experimental Psychology: Learning, Memory, and Cogni

Self Cite

View full text Add to dashboard Cite

show abstract

“…To alleviate these problems, the change process should be explicitly incorporated in the model. We have proposed to do this in a state–space model formalism (Speekenbrink & Shanks, 2008). The resulting dynamic lens model (DLM) consists of two dynamic linear models (e.g., West & Harrison, 1997).…”

Section: Dynamic Lens Model Analysismentioning

confidence: 99%

Learning in a changing environment.

Speekenbrink¹,

Shanks²

2010

Journal of Experimental Psychology: General

Self Cite

View full text Add to dashboard Cite

Multiple cue probability learning studies have typically focused on stationary environments. We present three experiments investigating learning in changing environments. A fine-grained analysis of the learning dynamics shows that participants were responsive to both abrupt and gradual changes in cue-outcome relations. We found no evidence that participants adapted to these types of change in qualitatively different ways. Also, in contrast to earlier claims that these tasks are learned implicitly, participants showed good insight into what they learned. By fitting formal learning models, we investigated whether participants learned global functional relationships or made localized predictions from similar experienced exemplars. Both a local (the Associative Learning Model) and a global learning model (the novel Bayesian Linear Filter) fitted the data of the first two experiments. However, the results of Experiment 3, which was specifically designed to discriminate between local and global learning models, provided more support for global learning models. Finally, we present a novel model to account for the cue competition effects found in previous research and displayed by some of our participants. Nothing endures but change Heraclitus (540 BC -480 BC).Predicting future events from past experience is a fundamental aspect of daily life. For instance, policy makers have to predict the outcome of interventions and sports coaches must

show abstract

“…Our task involved only deterministic rules, but included rules of different complexity, focusing on high-order dependencies between elements in the pattern. We extended ideas of feature-based learning (6,(11)(12)(13) to present a reinforcement learning-like family of maximum entropy-based models to describe how individuals learn different high-order classification rules. We found that these models capture individual behavior to a high degree of accuracy, and can also predict individual behavior.…”

mentioning

confidence: 99%

High-order feature-based mixture models of classification learning predict individual learning curves and enable personalized teaching

Cohen

Schneidman

2012

Proc. Natl. Acad. Sci. U.S.A.

View full text Add to dashboard Cite

Pattern classification learning tasks are commonly used to explore learning strategies in human subjects. The universal and individual traits of learning such tasks reflect our cognitive abilities and have been of interest both psychophysically and clinically. From a computational perspective, these tasks are hard, because the number of patterns and rules one could consider even in simple cases is exponentially large. Thus, when we learn to classify we must use simplifying assumptions and generalize. Studies of human behavior in probabilistic learning tasks have focused on rules in which pattern cues are independent, and also described individual behavior in terms of simple, single-cue, feature-based models. Here, we conducted psychophysical experiments in which people learned to classify binary sequences according to deterministic rules of different complexity, including high-order, multicue-dependent rules. We show that human performance on such tasks is very diverse, but that a class of reinforcement learning-like models that use a mixture of features captures individual learning behavior surprisingly well. These models reflect the important role of subjects' priors, and their reliance on high-order features even when learning a low-order rule. Further, we show that these models predict future individual answers to a high degree of accuracy. We then use these models to build personally optimized teaching sessions and boost learning.inference | information | maximum entropy W e regularly learn to classify sensory stimuli into novel categories, often using an impoverished sampling of the stimulus space and the underlying rule: even a "simple" task of classifying patterns of n bits into two categories, requires an implicit mapping of the 2 n possible patterns, which means there are 2 2 n potential deterministic classification rules. It is clear then that when we learn to classify, we cannot simply explore the space of rules and patterns, but instead must rely on simplifying assumptions.Analysis of human learning of deterministic classification rules has focused on modeling of the average behavior of subjects (1-3), and explored the effect of rule complexity on the average level of success (4). Learning to classify according to a probabilistic rule is inherently ambiguous, and so studies of such tasks have focused on simpler rules than those used in deterministic classification. For example, the weather prediction (WP) task (5) requires learning probabilistic associations between multiple cues and a label, where each cue carries independent information about the correct label. Analysis of the learning strategy of individual subjects in this task has compared single-cue or single feature-based strategies, and the possibility of switching between such strategies (6, 7). Associative or Bayesian learning models that rely on simple stimulus features were used to describe the diversity of individual learning dynamics that subjects exhibited and compare between subjects (8), and reflected differences between healthy sub...

show abstract

Through the looking glass: a dynamic lens model approach to multiple cue probability learning

Cited by 5 publications

References 16 publications

Putting bandits into context: How function learning supports decision making.

Putting bandits into context: How function learning supports decision making.

Learning in a changing environment.

High-order feature-based mixture models of classification learning predict individual learning curves and enable personalized teaching

Contact Info

Product

Resources

About