Examining performance and likelihood ratios for two likelihood ratio systems using the PROVEDIt dataset

Abstract

A likelihood ratio (LR) system is defined as the entire pipeline of the measurement and interpretation processes where probabilistic genotyping software (PGS) is a piece of the whole LR system. To gain understanding on how two LR systems perform, a total of 154 two-person, 147 three-person, and 127 four-person mixture profiles of varying DNA quality, DNA quantity, and mixture ratios were obtained from the filtered (.CSV) files of the GlobalFiler 29 cycles 15s PROVEDIt dataset and deconvolved in two independently developed fully continuous programs, STRmix v2.6 and EuroForMix v2.1.0. Various parameters were set in each software and LR computations obtained from the two software were based on same/fixed EPG features, same pair of propositions, number of contributors, theta, and population allele frequencies. The ability of each LR system to discriminate between contributor (H1-true) and non-contributor (H2-true) scenarios was evaluated qualitatively and quantitatively. Differences in the numeric LR values and their corresponding verbal classifications between the two LR systems were compared. The magnitude of the differences in the assigned LRs and the potential explanations for the observed differences greater than or equal to 3 on the log10 scale were described. Cases of LR < 1 for H1-true tests and LR > 1 for H2-true tests were also discussed. Our intent is to demonstrate the value of using a publicly available ground truth known mixture dataset to assess discrimination performance of any LR system and show the steps used to understand similarities and differences between different LR systems. We share our observations with the forensic community and describe how examining more than one PGS with similar discrimination power can be beneficial, help analysts compare interpretation especially with low-template profiles or minor contributor cases, and be a potential additional diagnostic check even if software in use does contain certain diagnostic statistics as part of the output.

Hamiltonian Monte Carlo with strict convergence criteria reduces run-to-run variability in forensic DNA mixture deconvolution

“…We quantify these aspects for our proposed solution on publicly available test mixtures from previously published benchmarks: the ProvedIt dataset [34] and the MIX05 and MIX13 studies [18]. For the GlobalFiler™ mixtures from the ProvedIt dataset, we use the hyper-parametrisation suggested by Riman et al [34]. For the MIX05 and MIX13 datasets, we use the hyper-parametrisation from Buckleton et al [16].…”

“…Moreover, it should be precise and have low inference runtime. We quantify these aspects for our proposed solution on publicly available test mixtures from previously published benchmarks: the ProvedIt dataset [34] and the MIX05 and MIX13…”

“…In future work we plan to compare our solution against other algorithms on the proposed ProvedIt benchmarks (e.g. [18, 32]), provide comparative analysis on two main approaches algorithmic approaches (Bayesian inference and maximum likelihood estimation) when the same probabilistic model is used and work on further improvements of the precision of the model.…”

“…STRmix™ showed examples with a 10-fold difference between two runs [9, 12]. Some papers argue that a remedy to decreased precision is an increase of MCMC iterations [12, 32], but this involves additional computational cost.…”

High‐quality data from a forensically relevant single‐cell pipeline enabled by low PBS and proteinase K concentrations

“…If a partial set of assumptions is applied, for example, the weight of evidence can be incomplete, possibly significantly so [7]. Thus, despite computational advances interpreting forensic DNA mixture samples remains arduous, requiring substantive resources to validate [8], train personnel [9], and support.…”