Prior Knowledge Elicitation: The Past, Present, and Future

Mikkola, Petrus; Martin, Osvaldo A.; Chandramouli, Suyog; Hartmann, Marcelo; Abril, Oriol; Thomas, Owen; Pesonen, Henri; Corander, Jukka; Vehtari, Aki; Kaski, Samuel; Bürkner, Paul‐Christian; Klami, Arto

doi:10.1214/23-ba1381

Cited by 12 publications

(8 citation statements)

References 110 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The advocated use of weakly prior exposes the practitioner to the arbitrariness, a classical point for criticism. The authors did an important contribution in this paper and they rise, in an underlying manner, many questions that inspire more research and more work on this topic which remains a current issue 4 …”

Section: Likelihood Ratio CI [026 09999]mentioning

confidence: 99%

Discussion of Specifying prior distributions in reliability applications

Gouno

2023

Appl Stoch Models Bus & Ind

View full text Add to dashboard Cite

Section: Likelihood Ratio CI [026 09999]mentioning

confidence: 99%

Discussion of Specifying prior distributions in reliability applications

Gouno

2023

Appl Stoch Models Bus & Ind

View full text Add to dashboard Cite

“…In addition to formalizing our theories as simulator models, we need to complete this step by defining prior distributions over the (free) model parameters, as would be required for any analysis. We point the reader to Wilson and Collins, 2019 ; Schad et al, 2021 ; Mikkola et al, 2021 for guidance on specifying prior distributions, as this is a broad research topic and a detailed treatment is beyond the scope of this work. Generally, prior distributions should reflect what is known about the model parameters prior to running the experiment, based on either empirical data or general domain knowledge.…”

Section: Step 2: Cast Your Theory As a Computational Modelmentioning

confidence: 99%

Designing optimal behavioral experiments using machine learning

Valentin,

Kleinegesse,

Bramley

et al. 2024

eLife

View full text Add to dashboard Cite

Computational models are powerful tools for understanding human cognition and behavior. They let us express our theories clearly and precisely and offer predictions that can be subtle and often counter-intuitive. However, this same richness and ability to surprise means our scientific intuitions and traditional tools are ill-suited to designing experiments to test and compare these models. To avoid these pitfalls and realize the full potential of computational modeling, we require tools to design experiments that provide clear answers about what models explain human behavior and the auxiliary assumptions those models must make. Bayesian optimal experimental design (BOED) formalizes the search for optimal experimental designs by identifying experiments that are expected to yield informative data. In this work, we provide a tutorial on leveraging recent advances in BOED and machine learning to find optimal experiments for any kind of model that we can simulate data from, and show how by-products of this procedure allow for quick and straightforward evaluation of models and their parameters against real experimental data. As a case study, we consider theories of how people balance exploration and exploitation in multi-armed bandit decision-making tasks. We validate the presented approach using simulations and a real-world experiment. As compared to experimental designs commonly used in the literature, we show that our optimal designs more efficiently determine which of a set of models best account for individual human behavior, and more efficiently characterize behavior given a preferred model. At the same time, formalizing a scientific question such that it can be adequately addressed with BOED can be challenging and we discuss several potential caveats and pitfalls that practitioners should be aware of. We provide code to replicate all analyses as well as tutorial notebooks and pointers to adapt the methodology to different experimental settings.

show abstract

“…Specifying useful priors is a central aspect of Bayesian statistics (Gelman et al, 2013;Martin et al, 2021;Martin & Teste, 2022), yet prior elicitation techniques that would make this step easier and more systematic are not routinely used within practical Bayesian workflows (Mikkola et al, 2023). Instead, practitioners typically rely on ad hoc procedures based on a mix of their own experience and recommendations available in the literature, which in general has not been systematized (Sarma & Kay, 2020).…”

Section: Statement Of Needmentioning

confidence: 99%

“…Instead, practitioners typically rely on ad hoc procedures based on a mix of their own experience and recommendations available in the literature, which in general has not been systematized (Sarma & Kay, 2020). One reason is that current solutions are simply not sufficient for practical data analysis and do not integrate well with probabilistic programming libraries (Mikkola et al, 2023). PreliZ is a library for prior elicitation that aims to facilitate the task for practitioners by offering a set of tools for the various facets of prior elicitation.…”

Section: Statement Of Needmentioning

confidence: 99%

PreliZ: A tool-box for prior elicitation

Icazatti,

Abril-Pla,

Klami

et al. 2023

JOSS

Self Cite

View full text Add to dashboard Cite

In a Bayesian modeling workflow, a prior distribution can be chosen in different ways as long as it captures the uncertainty about model parameters prior to observing any data. Particularly, prior elicitation refers to the process of transforming the knowledge of a particular domain into well-defined probability distributions. Here we introduce PreliZ, a Python package aimed at helping practitioners to choose prior distributions.

show abstract

Prior Knowledge Elicitation: The Past, Present, and Future

Cited by 12 publications

References 110 publications

Discussion of Specifying prior distributions in reliability applications

Discussion of Specifying prior distributions in reliability applications

Designing optimal behavioral experiments using machine learning

PreliZ: A tool-box for prior elicitation

Contact Info

Product

Resources

About