How the Choice of Distance Measure Influences the Detection of Prior-Data Conflict

Lek, Kimberley; Schoot, Rens van de

doi:10.3390/e21050446

Cited by 8 publications

(5 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…An alternative criterion has been developed 67 that computes whether the distance between the prior and the data is unexpected. For a comparison of both criteria, we direct the reader to Lek and van de Schoot 68 .…”

Section: Prior Predictive Distributionmentioning

confidence: 99%

Bayesian statistics and modelling

Schoot

Depaoli

King

et al. 2021

Nat Rev Methods Primers

551

381

View full text Add to dashboard Cite

show abstract

Section: Prior Predictive Distributionmentioning

confidence: 99%

Bayesian statistics and modelling

Schoot

Depaoli

King

et al. 2021

Nat Rev Methods Primers

551

381

View full text Add to dashboard Cite

show abstract

“…Quantifying sensitivity based on the distance between the base posterior and perturbed posteriors has been previously considered (Al-Labadi & Wang, 2019;Kurtek & Bharath, 2015;O'Hagan, 2003). In principle, many different divergence measures could be used, although there may be slight differences in interpretation (see, for example Lek & Van De Schoot, 2019), however, the cumulative Jensen-Shannon divergence (CJS; Nguyen & Vreeken, 2015) has two properties that make it appropriate for our use case. First, its symmetrised form is upper-bounded, like the standard Jensen-Shannon divergence (Lin, 1991), which aids interpretation.…”

Section: Distance-based Sensitivitymentioning

confidence: 99%

Detecting and diagnosing prior and likelihood sensitivity with power-scaling

Kallioinen¹,

Paananen²,

Bürkner³

et al. 2021

Preprint

View full text Add to dashboard Cite

Determining the sensitivity of the posterior to perturbations of the prior and likelihood is an important part of the Bayesian workflow. We introduce a practical and computationally efficient sensitivity analysis approach that is applicable to a wide range of models, based on power-scaling perturbations. We suggest a diagnostic based on this that can indicate the presence of prior-data conflict or likelihood noninformativity. The approach can be easily included in Bayesian workflows with minimal work by the model builder. We present the implementation of the approach in our new R package priorsense and demonstrate the workflow on case studies of real data.

show abstract

“…Since the results of both measures are highly comparable, see Figure 4 ; the results section below presents only the detailed PPC results. For a comparison between the two methods, see Lek and Van De Schoot (2019) . The complete results, including annotated syntax, can be found on OSF (see text footnote 1).…”

Section: Study 2–expert Elicitation and Prior-data Conflictsmentioning

confidence: 99%

The Use of Questionable Research Practices to Survive in Academia Examined With Expert Elicitation, Prior-Data Conflicts, Bayes Factors for Replication Effects, and the Bayes Truth Serum

et al. 2021

View full text Add to dashboard Cite

The popularity and use of Bayesian methods have increased across many research domains. The current article demonstrates how some less familiar Bayesian methods can be used. Specifically, we applied expert elicitation, testing for prior-data conflicts, the Bayesian Truth Serum, and testing for replication effects via Bayes Factors in a series of four studies investigating the use of questionable research practices (QRPs). Scientifically fraudulent or unethical research practices have caused quite a stir in academia and beyond. Improving science starts with educating Ph.D. candidates: the scholars of tomorrow. In four studies concerning 765 Ph.D. candidates, we investigate whether Ph.D. candidates can differentiate between ethical and unethical or even fraudulent research practices. We probed the Ph.D.s’ willingness to publish research from such practices and tested whether this is influenced by (un)ethical behavior pressure from supervisors or peers. Furthermore, 36 academic leaders (deans, vice-deans, and heads of research) were interviewed and asked to predict what Ph.D.s would answer for different vignettes. Our study shows, and replicates, that some Ph.D. candidates are willing to publish results deriving from even blatant fraudulent behavior–data fabrication. Additionally, some academic leaders underestimated this behavior, which is alarming. Academic leaders have to keep in mind that Ph.D. candidates can be under more pressure than they realize and might be susceptible to using QRPs. As an inspiring example and to encourage others to make their Bayesian work reproducible, we published data, annotated scripts, and detailed output on the Open Science Framework (OSF).

show abstract

How the Choice of Distance Measure Influences the Detection of Prior-Data Conflict

Cited by 8 publications

References 22 publications

Bayesian statistics and modelling

Bayesian statistics and modelling

Detecting and diagnosing prior and likelihood sensitivity with power-scaling

The Use of Questionable Research Practices to Survive in Academia Examined With Expert Elicitation, Prior-Data Conflicts, Bayes Factors for Replication Effects, and the Bayes Truth Serum

Contact Info

Product

Resources

About