Measuring coherence of computer-assisted likelihood ratio methods

Haraksim, Rudolf; Ramos, Daniel; Meuwly, Didier; Berger, Charles E.H.

doi:10.1016/j.forsciint.2015.01.033

Cited by 22 publications

(18 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This letter proposes to depict aggregated levels of decision risk on ROCCHs with respect to verbal scales, assuming optimal calibration. 9 Whereas, calibration performance is conventionally depicted by applied probability of error and empirical cross-entropy plots [7], [8].…”

Section: Discussionmentioning

confidence: 99%

“…The authors would like to thanks to E. Tabassi for discussions. 9 As indicated in Fig. 4, calibration performance can be depicted as well.…”

Section: Acknowledgmentmentioning

confidence: 98%

“…. , +3, second agree on a range of considerable priors, 8 then derive dependent costs, cf., (4), and third, adjust the threshold depending on the representative operating point by utilizing (9), when considering C II = 1, e.g., the C I cost proposed by the representative operating point may vary in a (−10%, +15%) band, or need to be downscaled to a distinct C I . By adjusting thresholds, other verbal bands can be reached, e.g., a threshold of scale +2 increasing to scale +3.…”

Section: B Verbal Bands: Representative Operating Pointsmentioning

confidence: 99%

See 2 more Smart Citations

Making Likelihood Ratios Digestible for Cross-Application Performance Assessment

Nautsch

Meuwly

Ramos

et al. 2017

IEEE Signal Process. Lett.

Self Cite

View full text Add to dashboard Cite

Performance estimation is crucial to the assessment of novel algorithms and systems. In detection error tradeoff (DET) diagrams, discrimination performance is solely assessed targeting one application, where cross-application performance considers risks resulting from decisions, depending on application constraints. For the purpose of interchangeability of research results across different application constraints, we propose to augment DET curves by depicting systems regarding their support of security and convenience levels. Therefore, application policies are aggregated into levels based on verbal likelihood ratio scales, providing an easy to use concept for business-to-business communication to denote operative thresholds. We supply a reference implementation in Python, an exemplary performance assessment on synthetic score distributions, and a fine-tuning scheme for Bayes decision thresholds, when decision policies are bounded rather than fix. Index Terms-Bayes decision framework, biometric verification, binary decisions, detection error tradeoff (DET), verbal scales.

show abstract

Section: Discussionmentioning

confidence: 99%

“…The authors would like to thanks to E. Tabassi for discussions. 9 As indicated in Fig. 4, calibration performance can be depicted as well.…”

Section: Acknowledgmentmentioning

confidence: 98%

Section: B Verbal Bands: Representative Operating Pointsmentioning

confidence: 99%

See 1 more Smart Citation

Making Likelihood Ratios Digestible for Cross-Application Performance Assessment

Nautsch

Meuwly

Ramos

et al. 2017

IEEE Signal Process. Lett.

Self Cite

View full text Add to dashboard Cite

show abstract

“…Based on its improved performance and computational simplicity, the multimodal LR method was used in [4] to evaluate the coherence of the scores produced by an AFIS algorithm. The proposed approach can be used in cases where multimodal score distributions are observed, even in cases in which discrete score regions are presented.…”

Section: Discussionmentioning

confidence: 99%

“…The accuracy was defined in [4,19] as the closeness of agreement between a LR computed by a given method and the ground truth status of the proposition in a decision-theoretical inference model. The discriminating power was defined in [4,19] as the performance property representing the capability of a given LR method to distinguish amongst forensic comparisons where different propositions are true. As it can be seen in [7,[19][20][21], empirical cross-entropy (ECE) curves and log LR cost (Cllr) are increasingly accepted measures of accuracy, the latter being a summarising measure of the former.…”

Section: Performance Measuresmentioning

confidence: 99%

Validation of likelihood ratio methods for forensic evidence evaluation handling multimodal score distributions

2016

Self Cite

View full text Add to dashboard Cite

Esta es la versión de autor del artículo publicado en: This is an author produced version of a paper published in:IET Biometrics 2.6 (2016):61-69 DOI: http://dx.doi.org/10.1049/iet-bmt.2015.0059 Copyright: © 2016 The Institution of Engineering and TechnologyEl acceso a la versión del editor puede requerir la suscripción del recurso Access to the published version may require subscription ABSTRACT: This article presents a method for computing Likelihood Ratios (LR) from multimodal score distributions produced by an Automated Fingerprint Identification System (AFIS) feature extraction and comparison algorithm. The AFIS algorithm used to compare fingermarks and fingerprints was primarily developed for forensic investigation rather than for forensic evaluation. The computation of the scores is speed-optimized and performed on three different stages, each of which outputs discriminating scores of different magnitudes together forming a multimodal score distribution. It is worthy mentioning that each fingermark to fingerprint comparison performed by the AFIS algorithm results in one single similarity score (e.g. one score per comparison). The multimodal nature of the similarity scores can be typical for other biometric systems and the method proposed in this work can be applied in similar cases, where the multimodal nature in similarity scores is observed. In this work we address some of the problems related to modelling such distributions and propose solutions to issues like data sparsity, dataset shift and over-fitting. The issues mentioned affect the methods traditionally used in the situation when a multimodal nature in the similarity scores is observed (a Kernel Density Functions (KDF) was used to illustrate these issues in our case). Furthermore, the method proposed produces interpretable results in the situations when the similarity scores are sparse and traditional approaches lead to erroneous LRs of huge magnitudes.

show abstract

Ink source prediction and assessment based on direct analysis in real‐time mass spectrometry via the likelihood ratio

Zhang

Yang

Tang

et al. 2023

Journal of Chemometrics

View full text Add to dashboard Cite

Ink classification is the ability to distinguish unknown inks into different groups, and ink source prediction is the ability to predict the manufacturer or model of an unknown ink. These are regular tasks in forensic analysis. The latter is more challenging than the former, as ink source prediction has expanded beyond ink classification. In this work, we reported on an approach to predict the source of black inks based on direct analysis in real time mass spectrometry and assess the strength of black ink source prediction conclusion via the likelihood ratio, using a dataset that included 39 inks from three manufacturers with a high market share. Most of these inks contain similar or identical chemical components. Dimensionality reduction based on the principal component analysis and unified manifold approximation and projection algorithms was implemented, and subsequently, the distribution plots illustrated the variations between and within the inks. Unified manifold approximation and projection showed significant priority in avoiding overcrowding of cluster representation versus principal component analysis, with results as high as 99.83% for the prediction of the ink source using 41,432 spectra data (70% data for training and 30% data for testing) after dimensionality reduction. A likelihood ratio was used to evaluate the strength of ink evidence, and the pool‐adjacent‐violators algorithm and logistic algorithms were used to calibrate the likelihood ratio. The results showed that the pool‐adjacent‐violators algorithm and logistic algorithms both had an excellent equal error rate of 0.004 but slightly different results in the rates of misleading evidence in favor of the prosecutor's hypothesis, rates of misleading evidence in favor of the defense's hypothesis, log likelihood ratio costs after calibration ( Cllrcal), and minimum log likelihood ratio costs ( Cllrmin). A blind test validated the robustness of the methods.

show abstract

Measuring coherence of computer-assisted likelihood ratio methods

Cited by 22 publications

References 17 publications

Making Likelihood Ratios Digestible for Cross-Application Performance Assessment

Making Likelihood Ratios Digestible for Cross-Application Performance Assessment

Validation of likelihood ratio methods for forensic evidence evaluation handling multimodal score distributions

Ink source prediction and assessment based on direct analysis in real‐time mass spectrometry via the likelihood ratio

Contact Info

Product

Resources

About