Properties of Fisher information gain for Bayesian design of experiments

Overstall, Antony M.

doi:10.1016/j.jspi.2021.10.006

Cited by 4 publications

(5 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Metrics of optimality other than the EIG are also possible [9,20,120], with the BED framework more generally referring to any approach that optimizes an objective of the form E p(θ)p(y|θ,ξ) [U (ξ, θ, y)] for some utility, U , that is a functional of the posterior, p(θ|y, ξ) (with some authors further relaxing this constraint on the form of U ). For example, the notion of an expected Fisher information gain has also recently been considered [104,111,138,144] because it can be easily estimated without evaluating either the posterior or marginal likelihood. Our focus, though, will be on maximizing the EIG defined in (2), as this remains the most commonly used information-theoretic BED approach; we implicitly refer to this specific approach when using the term BED in the rest of the paper.…”

Section: Formalizing Information Gainmentioning

confidence: 99%

“…The BED literature is spread across a wide variety of different fields, with papers appearing in, for example, statistics [104,111], machine learning [12,32], engineering [5,18], physics [58], and social science [66,98] venues. This somewhat scattered nature has sometimes led to a, potentially problematic, detachment between research threads.…”

Section: Linking With Related Areasmentioning

confidence: 99%

See 1 more Smart Citation

Modern Bayesian Experimental Design

Rainforth¹,

Foster²,

Ivanova³

et al. 2023

Preprint

View full text Add to dashboard Cite

Bayesian experimental design (BED) provides a powerful and general framework for optimizing the design of experiments. However, its deployment often poses substantial computational challenges that can undermine its practical use. In this review, we outline how recent advances have transformed our ability to overcome these challenges and thus utilize BED effectively, before discussing some key areas for future development in the field.

show abstract

Section: Formalizing Information Gainmentioning

confidence: 99%

Section: Linking With Related Areasmentioning

confidence: 99%

Modern Bayesian Experimental Design

Rainforth¹,

Foster²,

Ivanova³

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…The optimal FIG design maximises the trace of Ī(τ ), which equals the sum of its eigenvalues. Often this sum is maximised when one eigenvalue is large and the others are small: we provide an example in Section 6, and also Overstall (2022) describes a linear model where this occurs. In a Bayesian setting, I(θ; τ ) is an approximation to posterior precision (see e.g.…”

Section: Lack Of Reparameterisation Invariancementioning

confidence: 99%

“…Some preprints of this paper used this name for Utrace instead. Our usage here is a closer analogy to Shannon information gain, and matches that ofOverstall (2022).…”

mentioning

confidence: 92%

Bayesian Experimental Design Without Posterior Calculations: An Adversarial Approach

2023

View full text Add to dashboard Cite

Most computational approaches to Bayesian experimental design require making posterior calculations repeatedly for a large number of potential designs and/or simulated datasets. This can be expensive and prohibit scaling up these methods to models with many parameters, or designs with many unknowns to select. We introduce an efficient alternative approach without posterior calculations, based on optimising the expected trace of the Fisher information, as discussed by Walker (2016). We illustrate drawbacks of this approach, including lack of invariance to reparameterisation and encouraging designs in which one parameter combination is inferred accurately but not any others. We show these can be avoided by using an adversarial approach: the experimenter must select their design while a critic attempts to select the least favourable parameterisation. We present theoretical properties of this approach and show it can be used with gradient based optimisation methods to find designs efficiently in practice.

show abstract

“…More recently, alternatives have been proposed that avoid explicit computation of a posterior. These include adversarial methods (Prangle et al, 2019;Overstall, 2022), and amortized methods (Foster et al, 2021;Ivanova et al, 2021).…”

Section: Related Workmentioning

confidence: 99%

Optimizing Sequential Experimental Design with Deep Reinforcement Learning

Blau¹,

Bonilla²,

Dezfouli³

et al. 2022

Preprint

View full text Add to dashboard Cite

Bayesian approaches developed to solve the optimal design of sequential experiments are mathematically elegant but computationally challenging. Recently, techniques using amortization have been proposed to make these Bayesian approaches practical, by training a parameterized policy that proposes designs efficiently at deployment time. However, these methods may not sufficiently explore the design space, require access to a differentiable probabilistic model and can only optimize over continuous design spaces. Here, we address these limitations by showing that the problem of optimizing policies can be reduced to solving a Markov decision process (MDP). We solve the equivalent MDP with modern deep reinforcement learning techniques. Our experiments show that our approach is also computationally efficient at deployment time and exhibits state-of-the-art performance on both continuous and discrete design spaces, even when the probabilistic model is a black box.

show abstract

Properties of Fisher information gain for Bayesian design of experiments

Cited by 4 publications

References 14 publications

Modern Bayesian Experimental Design

Modern Bayesian Experimental Design

Bayesian Experimental Design Without Posterior Calculations: An Adversarial Approach

Optimizing Sequential Experimental Design with Deep Reinforcement Learning

Contact Info

Product

Resources

About