On the Reconciliation of Probability Assessments

Lindley, D. V.; Tversky, Amos; Brown, Rex V.

doi:10.2307/2345078

Cited by 322 publications

(144 citation statements)

References 31 publications

(2 reference statements)

Supporting

Mentioning

144

Contrasting

Order By: Relevance

“…During decades, Bayesian statisticians have been seriously concerned about the elicitation of probabilistic assessments [10,16,17], which can be understood given the Bayesian interpretation of probability as a degree of belief [18,19]. In this topic of research, one of the main questions under study has always been the performance of the probabilistic assessments, that can be summarized as follows: if someone is eliciting probability assessments (according to a given model and data, or based on personal experience), how can we evaluate how they perform?…”

Section: Performance Of Probabilistic Assessments: Strictly Proper Scmentioning

confidence: 99%

Reliable support: Measuring calibration of likelihood ratios

Ramos

González-Rodríguez

2013

Forensic Science International

103

View full text Add to dashboard Cite

Calculation of likelihood ratios (LR) in evidence evaluation still presents major challenges in many forensic disciplines: for instance, an incorrect selection of databases, a bad choice of statistical models, low quantity and bad quality of the evidence are factors that may lead to likelihood ratios supporting the wrong proposition in a given case. However, measuring performance of LR values is not straightforward, and adequate metrics should be defined and used. With this objective, in this work we describe the concept of calibration, a property of a set of LR values. We highlight that some desirable behavior of LR values happens if they are well calibrated. Moreover, we propose a tool for representing performance, the Empirical Cross-Entropy (ECE) plot, showing that it can explicitly measure calibration of LR values. We finally describe some examples using speech evidence, where the usefulness of ECE plots and the measurement of calibration is shown.

show abstract

Section: Performance Of Probabilistic Assessments: Strictly Proper Scmentioning

confidence: 99%

Reliable support: Measuring calibration of likelihood ratios

Ramos

González-Rodríguez

2013

Forensic Science International

103

View full text Add to dashboard Cite

show abstract

“…Even if the expert were very consistent and gave reasonably coherent assessments, reconciliation would be needed for correcting small rounding errors. Lindley et al (1979) investigated the reconciliation of assessments that are inconsistent with the laws of probabilities (incoherent). They developed least-squares procedures as reconciliation tools that may be used for any expert's incoherent assessments.…”

Section: Encoding the Dirichlet Hyperparametersmentioning

confidence: 99%

“…(III) The precision of each p i , i.e. the inverse of its variance, can be used as a weight to reflect the expert's confidence about each of her assessments (Lindley et al 1979). These weights are used in a weighted least-squares procedure.…”

Section: Encoding the Dirichlet Hyperparametersmentioning

confidence: 99%

Eliciting Dirichlet and Gaussian copula prior distributions for multinomial models

Elfadaly

Garthwaite

2016

Stat Comput

View full text Add to dashboard Cite

In this paper, we propose novel methods of quantifying expert opinion about prior distributions for multinomial models. Two different multivariate priors are elicited using median and quartile assessments of the multinomial probabilities. First, we start by eliciting a univariate beta distribution for the probability of each category. Then we elicit the hyperparameters of the Dirichlet distribution, as a tractable conjugate prior, from those of the univariate betas through various forms of reconciliation using least-squares techniques. However, a multivariate copula function will give a more flexible correlation structure between multinomial parameters if it is used as their multivariate prior distribution. So, second, we use beta marginal distributions to construct a Gaussian copula as a multivariate normal distribution function that binds these marginals and expresses the dependence structure between them. The proposed method elicits a positive-definite correlation matrix of this Gaussian copula. The two proposed methods are designed to be used through interactive graphical software written in Java.

show abstract

“…Lindley et al (1979). Alternatively the elicitation can be done in such a way that the input must be consistent, where care must be taken not to make one outcome more salient (e.g.…”

Section: Eliciting Complementary Eventsmentioning

confidence: 99%

A penny for your thoughts: a survey of methods for eliciting beliefs

2014

View full text Add to dashboard Cite

Incentivized methods for eliciting subjective probabilities in economic experiments present the subject with risky choices or bets that encourage truthful reporting. We discuss the most prominent elicitation methods and their underlying assumptions, provide theoretical comparisons, and propose some extensions to the standard framework. In addition, we survey the empirical literature on the performance of these elicitation methods in actual experiments, considering also practical issues of implementation such as order effects, hedging, and different ways of presenting probabilities and payment schemes to experimental subjects. We end with some thoughts on the merits of using incentives for belief elicitation and some guidelines for implementation. JEL-codes: C83, C91, D83.

show abstract

On the Reconciliation of Probability Assessments

Cited by 322 publications

References 31 publications

Reliable support: Measuring calibration of likelihood ratios

Reliable support: Measuring calibration of likelihood ratios

Eliciting Dirichlet and Gaussian copula prior distributions for multinomial models

A penny for your thoughts: a survey of methods for eliciting beliefs

Contact Info

Product

Resources

About