Ensemble learning for score likelihood ratios under the common source problem

Abstract

Machine learning‐based score likelihood ratios (SLRs) have emerged as alternatives to traditional likelihood ratios and Bayes factors to quantify the value of evidence when contrasting two opposing propositions. When developing a conventional statistical model is infeasible, machine learning can be used to construct a (dis)similarity score for complex data and estimate the ratio of the conditional distributions of the scores. Under the common source problem, the opposing propositions address if two items come from the same source. To develop their SLRs, practitioners create datasets using pairwise comparisons from a background population sample. These comparisons result in a complex dependence structure that violates the independence assumption made by many popular methods. We propose a resampling step to remedy this lack of independence and an ensemble approach to enhance the performance of SLR systems. First, we introduce a source‐aware resampling plan to construct datasets where the independence assumption is met. Using these newly created sets, we train multiple base SLRs and aggregate their outputs into a final value of evidence. Our experimental results show that this ensemble SLR can outperform a traditional SLR approach in terms of the rate of misleading evidence and discriminatory power and present more consistent results.

Likelihood ratios for changepoints in categorical event data with applications in digital forensics

“…While there has been some work that examines verbal scales that correspond to particular numerical intervals of the LR [23][24][25][26][27], there is no single, generally agreed-upon scale. To present experimental LR results in the literature, one approach that has been proposed is to split the LR values into three regions: supports the defense proposition, supports the prosecution proposition, and limited evidence [28,29]. In an experimental evaluation setting where the ground truth is known,…”