p -Values should not be used for evaluating the strength of DNA evidence

Kruijver, Maarten; Meester, Ronald; Slooten, Klaas

doi:10.1016/j.fsigen.2015.01.005

Cited by 12 publications

(10 citation statements)

References 29 publications

(43 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Algorithms have been laid out for the computation of the LR distribution and the p- value [ 43 , 44 ], but the p- value has faced its share of criticism as a statistic to replace the LR [ 38 , 45 ]. In addition to enabling control of the FPR, a benefit of the p- value is that it can be used as an indicative tool while performing validation studies on a mixture interpretation system that computes the LR.…”

Section: Discussionmentioning

confidence: 99%

Four model variants within a continuous forensic DNA mixture interpretation framework: Effects on evidential inference and reporting

et al. 2018

View full text Add to dashboard Cite

Continuous mixture interpretation methods that employ probabilistic genotyping to compute the Likelihood Ratio (LR) utilize more information than threshold-based systems. The continuous interpretation schemes described in the literature, however, do not all use the same underlying probabilistic model and standards outlining which probabilistic models may or may not be implemented into casework do not exist; thus, it is the individual forensic laboratory or expert that decides which model and corresponding software program to implement. For countries, such as the United States, with an adversarial legal system, one can envision a scenario where two probabilistic models are used to present the weight of evidence, and two LRs are presented by two experts. Conversely, if no independent review of the evidence is requested, one expert using one model may present one LR as there is no standard or guideline requiring the uncertainty in the LR estimate be presented. The choice of model determines the underlying probability calculation, and changes to it can result in non-negligible differences in the reported LR or corresponding verbal categorization presented to the trier-of-fact. In this paper, we study the impact of model differences on the LR and on the corresponding verbal expression computed using four variants of a continuous mixture interpretation method. The four models were tested five times each on 101, 1-, 2- and 3-person experimental samples with known contributors. For each sample, LRs were computed using the known contributor as the person of interest. In all four models, intra-model variability increased with an increase in the number of contributors and with a decrease in the contributor’s template mass. Inter-model variability in the associated verbal expression of the LR was observed in 32 of the 195 LRs used for comparison. Moreover, in 11 of these profiles there was a change from LR > 1 to LR < 1. These results indicate that modifications to existing continuous models do have the potential to significantly impact the final statistic, justifying the continuation of broad-based, large-scale, independent studies to quantify the limits of reliability and variability of existing forensically relevant systems.

show abstract

Section: Discussionmentioning

confidence: 99%

Four model variants within a continuous forensic DNA mixture interpretation framework: Effects on evidential inference and reporting

et al. 2018

View full text Add to dashboard Cite

show abstract

“…This seems reasonable but it is an important reminder that the fraction of nondonors giving an LR greater than or equal to the LR of the POI (termed the p ‐value) has been incorrectly suggested as a substitute for the LR itself. This concept was elegantly rebutted by Kruijver et al . If the p ‐value is used in place of the LR , it would always overstate the value of the evidence.…”

Section: Resultsmentioning

confidence: 99%

“…Most criticism of current methods for communicating LRs suggests that a frequency, and especially the risk of adventitious matches, would be either a preferable alternative or a useful supplement . Such a frequentist rendering would challenge many adherents of Bayesian logic , including ourselves. Between the authors, we have attempted to explain the LR in court in terms of prior odds, likelihood ratio, and posterior odds.…”

Section: Glossarymentioning

confidence: 99%

Using the Nondonor Distribution to Improve Communication and Inform Decision Making for Low LRs from Minor Contributors in Mixed DNA Profiles

Schuerman

Kalafut

Buchanan

et al. 2020

Journal of Forensic Sciences

View full text Add to dashboard Cite

The reporting of a likelihood ratio (LR) calculated from probabilistic genotyping software has become more popular since 2015 and has allowed for the use of more complex mixtures at court. The meaning of “inconclusive” LRs and how to communicate the significance of low LRs at court is now important. We present a method here using the distribution of LRs obtained from nondonors. The nondonor distribution is useful for examining calibration and discrimination for profiles that have produced LRs less than about 104. In this paper, a range of mixed DNA profiles of varying quantity were constructed and the LR distribution considering the minor contributor for a number of nondonors was compared to the expectation given a calibrated system. It is demonstrated that conditioning genotypes should be used where reasonable given the background information to decrease the rate of nondonor LRs above 1. In all 17 cases examined, the LR for the minor donor was higher than the nondonor LRs, and in 12 of the 17 cases, the 99.9 percentile of the nondonor distribution was lower when appropriate conditioning information was used. The output of the tool is a graph that can show the position of the LR for the person of interest set against the nondonor LR distribution. This may assist communication between scientists and the court.

show abstract

“…Or even: if this person of interest's DNA is mixed up with that of others, what likelihood ratios do we get in favour of his contribution, i.e., how well can we recover his DNA profile? These questions lead to different 'randomizations' of the LR (see also [11] for examples of this in the kinship setting).…”

Section: Likelihood Ratios and Exclusion Probabilitiesmentioning

confidence: 99%

Exclusion probabilities and likelihood ratios with applications to mixtures

Slooten

Egeland

2015

Int J Legal Med

View full text Add to dashboard Cite

The statistical evidence obtained from mixed DNA profiles can be summarised in several ways in forensic casework including the likelihood ratio (LR) and the Random Man Not Excluded (RMNE) probability. The literature has seen a discussion of the advantages and disadvantages of likelihood ratios and exclusion probabilities, and part of our aim is to bring some clarification to this debate. In a previous paper, we proved that there is a general mathematical relationship between these statistics: RMNE can be expressed as a certain average of the LR, implying that the expected value of the LR, when applied to an actual contributor to the mixture, is at least equal to the inverse of the RMNE. While the mentioned paper presented applications for kinship problems, the current paper demonstrates the relevance for mixture cases, and for this purpose, we prove some new general properties. We also demonstrate how to use the distribution of the likelihood ratio for donors of a mixture, to obtain estimates for exceedance probabilities of the LR for non-donors, of which the RMNE is a special case corresponding to L R>0. In order to derive these results, we need to view the likelihood ratio as a random variable. In this paper, we describe how such a randomization can be achieved. The RMNE is usually invoked only for mixtures without dropout. In mixtures, artefacts like dropout and drop-in are commonly encountered and we address this situation too, illustrating our results with a basic but widely implemented model, a so-called binary model. The precise definitions, modelling and interpretation of the required concepts of dropout and drop-in are not entirely obvious, and we attempt to clarify them here in a general likelihood framework for a binary model.

show abstract

p -Values should not be used for evaluating the strength of DNA evidence

Cited by 12 publications

References 29 publications

Four model variants within a continuous forensic DNA mixture interpretation framework: Effects on evidential inference and reporting

Four model variants within a continuous forensic DNA mixture interpretation framework: Effects on evidential inference and reporting

Using the Nondonor Distribution to Improve Communication and Inform Decision Making for Low LRs from Minor Contributors in Mixed DNA Profiles

Exclusion probabilities and likelihood ratios with applications to mixtures

Contact Info

Product

Resources

About