Lay understanding of forensic statistics: Evaluation of random match probabilities, likelihood ratios, and verbal equivalents.

Abstract

Forensic scientists have come under increasing pressure to quantify the strength of their evidence, but it is not clear which of several possible formats for presenting quantitative conclusions will be easiest for lay people, such as jurors, to understand. This experiment examined the way that people recruited from Amazon's Mechanical Turk (n ϭ 541) responded to 2 types of forensic evidence-a DNA comparison and a shoeprint comparison-when an expert explained the strength of this evidence 3 different ways: using random match probabilities (RMPs), likelihood ratios (LRs), or verbal equivalents of likelihood ratios (VEs). We found that verdicts were sensitive to the strength of DNA evidence regardless of how the expert explained it, but verdicts were sensitive to the strength of shoeprint evidence only when the expert used RMPs. The weight given to DNA evidence was consistent with the predictions of a Bayesian network model that incorporated the perceived risk of a false match from 3 causes (coincidence, a laboratory error, and a frame-up), but shoeprint evidence was undervalued relative to the same Bayesian model. Fallacious interpretations of the expert's testimony (consistent with the source probability error and the defense attorney's fallacy) were common and were associated with the weight given to the evidence and verdicts. The findings indicate that perceptions of forensic science evidence are shaped by prior beliefs and expectations as well as expert testimony and consequently that the best way to characterize and explain forensic evidence may vary across forensic disciplines.

DNA commission of the International society for forensic genetics: Assessing the value of forensic biological evidence - Guidelines highlighting the importance of propositions

“…There appears to have been no advice in the judgement what to do with this figure of 1 billion. The word 'match' invites the prosecutor's fallacy as lay-persons can easily believe that a 'match' is synonymous to 'identification' [80]. There is the implicit risk of confirmation bias where evidence inconvenient to the prosecution is ignored or underweighted [81] and there is the associated danger that apparently compelling 'evidence' may be used to infer the activity (i.e.…”

On Juror Decision Making: An Empathic Inquiry

“…These inverse fallacies can take the form of equating likelihood ratios with the probability that the defendant is guilty (the prosecutor's fallacy) or of understanding the likelihood ratio to indicate that a sizable number of people could have been the source of the evidence (the defense attorney's fallacy). Second, people find it difficult to properly comprehend probabilistic evidence, especially for extreme values, and seem insufficiently sensitive to those values when adjusting their posterior judgments of guilt (Koehler 2012, Thompson & Newman 2015, Thompson et al 2018.…”

Deconstructing Cross-Entropy for Probabilistic Binary Classifiers

“…However, in this case, this is even more important because, in a forensic case, end-users are typically not proficient in mathematics and statistics, and care should be taken in order to explain the performance results if requested on trial (in any case, the authors discourage the use of complex performance representations on trial given the difficulties of human beings in general and triers of fact in particular to understand forensic statistics. A recent study about this can be found in [ 40 ]).…”