Optimal experimental design for a class of bandit problems

Zhang, Shunan; Lee, Michael

doi:10.1016/j.jmp.2010.08.002

Cited by 19 publications

(11 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…These diagnoses in turn can help guide the decisions involved in experimental design. An extreme form of using models to guide experimental design involves the growing area of "optimal experimental design," in which the predictions made by competing models are used to choose the conditions presented in an experiment, or even the stimuli presented on a trial-by-trial basis (Cavagnaro, Pitt, Gonzalez, & Myung, 2013;Zhang & Lee, 2010).…”

Section: Making Modeling Robustmentioning

confidence: 99%

Robust Modeling in Cognitive Science

et al. 2019

Self Cite

View full text Add to dashboard Cite

In an attempt to increase the reliability of empirical findings, psychological scientists have recently proposed a number of changes in the practice of experimental psychology. Most current reform efforts have focused on the analysis of data and the reporting of findings for empirical studies. However, a large contingent of psychologists build models that explain psychological processes and test psychological theories using formal psychological models. Some, but not all, recommendations borne out of the broader reform movement bear upon the practice of behavioral or cognitive modeling. In this article, we consider which aspects of the current reform movement are relevant to psychological modelers, and we propose a number of techniques and practices aimed at making psychological modeling more transparent, trusted, and robust.Cognitive modeling | Reproducibility | Open science | Robustness | Model comparison You never want a serious crisis to go to waste . . . This crisis provides the opportunity for us to do things that you could not before.

show abstract

Section: Making Modeling Robustmentioning

confidence: 99%

Robust Modeling in Cognitive Science

et al. 2019

Self Cite

View full text Add to dashboard Cite

show abstract

“…These three questionnaires have been used in a variety of contexts and have been found to be reliable in measuring people's risk propensity (Harrison, Young, Butow, Salkeld, & Solomon, 2005). The decision-making tasks we consider are the BART, the preferential choice gambling task, the optimal stopping problem (Goldstein, McAfee, Suri, & Wright, 2020;Guan, Lee, & Vandekerckhove, 2015;Guan & Lee, 2018;Lee, 2006;Seale & Rapoport, 2000), and the bandit problem (Lee, Zhang, Munro, & Steyvers, 2011;Steyvers, Lee, & Wagenmakers, 2009;Zhang & Lee, 2010b). All four of these decision-making tasks involve risk and uncertainty, and have corresponding cognitive models with parameters that can be interpreted as measuring some form of risk propensity.…”

mentioning

confidence: 99%

A cognitive modeling analysis of risk in sequential choice tasks

Guan¹,

Stokes²,

Vandekerckhove³

et al. 2020

Preprint

View full text Add to dashboard Cite

There are many ways to measure how people manage risk when they make decisions. A standard approach is to measure risk propensity using self-report questionnaires. An alternative approach is to use decision-making tasks that involve risk and uncertainty, and apply cognitive models of task behavior to infer parameters that measure people’s risk propensity. We report the results of a within-participants experiment that used three questionnaires and four decision-making tasks. The questionnaires are the Risk Propensity Scale, the Risk Taking Index, and the DomainSpecific Risk Taking Scale. The decision-making tasks are the Balloon Analogue Risk Task, the preferential choice gambling task, the optimal stopping problem, and the bandit problem. We analyze the relationships between the risk measures and cognitive parameters using Bayesian inferences about the patterns of correlation, and using a novel cognitive latent variable modeling approach. The results show that people’s risk propensity is generally consistent within different conditions for each of the decision-making tasks. There is, however, little evidence that the way people manage risk generalizes across the tasks, or that it corresponds to the questionnaire measures.

show abstract

“…In general, optimizing MDPs may be computationally challenging, as the design has to specify transition probabilities and rewards for each state-action pair, possibly resulting in a highdimensional problem. However, tractable optimization problems may also be obtained for experiments that can be represented using highly structured MDPs, with only few tunable design variables [20,54]. We expect that the crucial challenge for future work will be to elucidate general correspondences between model classes and experimental structures that allow their discrimination, while still being amenable to optimization (i.e., resulting in low-dimensional problems).…”

Section: Limitations and Future Workmentioning

confidence: 99%

Computational optimization of associative learning experiments

Melinščak

Bach

2020

PLoS Comput Biol

View full text Add to dashboard Cite

With computational biology striving to provide more accurate theoretical accounts of biological systems, use of increasingly complex computational models seems inevitable. However, this trend engenders a challenge of optimal experimental design: due to the flexibility of complex models, it is difficult to intuitively design experiments that will efficiently expose differences between candidate models or allow accurate estimation of their parameters. This challenge is well exemplified in associative learning research. Associative learning theory has a rich tradition of computational modeling, resulting in a growing space of increasingly complex models, which in turn renders manual design of informative experiments difficult. Here we propose a novel method for computational optimization of associative learning experiments. We first formalize associative learning experiments using a low number of tunable design variables, to make optimization tractable. Next, we combine simulation-based Bayesian experimental design with Bayesian optimization to arrive at a flexible method of tuning design variables. Finally, we validate the proposed method through extensive simulations covering both the objectives of accurate parameter estimation and model selection. The validation results show that computationally optimized experimental designs have the potential to substantially improve upon manual designs drawn from the literature, even when prior information guiding the optimization is scarce. Computational optimization of experiments may help address recent concerns over reproducibility by increasing the expected utility of studies, and it may even incentivize practices such as study pre-registration, since optimization requires a pre-specified analysis plan. Moreover, design optimization has the potential not only to improve basic research in domains such as associative learning, but also to play an important role in translational research. For example, design of behavioral and physiological diagnostic tests in the nascent field of computational psychiatry could benefit from an optimization-based approach, similar to the one presented here.

show abstract

Optimal experimental design for a class of bandit problems

Cited by 19 publications

References 27 publications

Robust Modeling in Cognitive Science

Robust Modeling in Cognitive Science

A cognitive modeling analysis of risk in sequential choice tasks

Computational optimization of associative learning experiments

Contact Info

Product

Resources

About