Quantifying the accuracy of forensic examiners in the absence of a “gold standard”.

Mossman, Douglas; Bowen, Michael D.; Vanness, David J.; Bienenfeld, David; Correll, Terry; Kay, Jerald; Klykylo, William M.; Lehrer, Douglas S.

doi:10.1007/s10979-009-9197-5

Cited by 21 publications

(29 citation statements)

References 56 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Information criteria have found use in the accuracy literature primarily as tool for choosing between multiple dependence structures. In a typical application, a subset of all possible covariance structures between a collection of tests is decided in advance, the model resulting from each choice is determined, and the criteria then used to select candidates [18,34,41,76,85,132,180,181]. Other applications include analysis of prior information as in Table IV [74,75], fine-tuning time-dependence effects [139,146], or choosing between nonignorable missing data models [182].…”

Section: Finding the Best Fit: Model Selectionmentioning

confidence: 99%

Estimation of diagnostic test accuracy without full verification: a review of latent class methods

Collins

Huynh

2014

Statist. Med.

View full text Add to dashboard Cite

The performance of a diagnostic test is best evaluated against a reference test that is without error. For many diseases, this is not possible, and an imperfect reference test must be used. However, diagnostic accuracy estimates may be biased if inaccurately verified status is used as the truth. Statistical models have been developed to handle this situation by treating disease as a latent variable. In this paper, we conduct a systematized review of statistical methods using latent class models for estimating test accuracy and disease prevalence in the absence of complete verification.

show abstract

Section: Finding the Best Fit: Model Selectionmentioning

confidence: 99%

Estimation of diagnostic test accuracy without full verification: a review of latent class methods

Collins

Huynh

2014

Statist. Med.

View full text Add to dashboard Cite

show abstract

“…Finally, two publications by Mossman (; Mossman et al, ) examined the potential for mathematical analysis of AC cases. In the first article, Mossman considered Buchanan's () position regarding proportionality (see discussion in “Conceptual Guidelines”).…”

Section: Interpretive Issuesmentioning

confidence: 99%

“…Mossman arrived at four factors that he believed comprised an AC opinion, which are susceptible to bias and could be accounted for mathematically to consider the accuracy of an AC opinion. In the second article, Mossman et al () applied the mathematical concepts, as well as latent class models, to quantify the accuracy of expert opinions regarding AC. Five board‐certified psychiatrists reviewed 156 reports and used a five‐point scale to assign a score of the defendant's AC‐related abilities and overall AC status.…”

Section: Interpretive Issuesmentioning

confidence: 99%

Ten Year Research Update (2001–2010): Evaluations for Competence to Stand Trial (Adjudicative Competence)

Fogel

Schiffman

Mumley

et al. 2013

Behavioral Sci & The Law

View full text Add to dashboard Cite

This article reviews and evaluates publications during 2001-2010 with relevance for assessments of competence to stand trial, referred to in this article as adjudicative competence. The review focuses specifically on articles that provide new concepts or data supported by research or case analyses. The studies are reviewed under the following headings: (i) systemic issues, (ii) conceptual guidelines for AC evaluations, (iii) AC assessment methods, (iv) empirical correlates of AC judgments and psycholegal abilities, (v) quality of AC evaluations and reports, (vi) interpretive issues, (vii) special populations (defendants who are elderly, defendants with intellectual disabilities), (viii) AC evaluations of juveniles, and (ix) treatment of incompetent defendants. Suggestions are offered for further research to advance the quality of clinical evaluations of adjudicative competence.

show abstract

“…Implementing the LCM approach involves evaluating the same cases with multiple diagnostic modalities, which often permits statistical identification of models that include accuracy parameters for those modalities. Mossman et al (2010) described the potential usefulness of this approach in examining assessment of adjudicative competence and showed that evaluators who based competence judgments on written reports appeared to be highly accurate. In this study, we investigated whether LCM methods could use data from real-life subjects evaluated in a forensic context to generate inferences about the accuracy of cognitive effort tests used to evaluate potential neuropsychological malingering.…”

mentioning

confidence: 99%

Estimating the accuracy of neurocognitive effort measures in the absence of a “gold standard”.

Mossman

Wygant

Gervais³

2012

Psychological Assessment

Self Cite

View full text Add to dashboard Cite

Psychologists frequently use symptom validity tests (SVTs) to help determine whether evaluees' test performance or reported symptoms accurately represent their true functioning and capability. Most studies evaluating the accuracy of SVTs have used either known-group comparisons or simulation designs, but these approaches have well-known limitations (potential misclassifications or lack of ecological validity). This study uses latent class modeling (LCM) implemented in a Bayesian framework to estimate SVT classification accuracy based on data obtained from real-life forensic evaluations. We obtained archival data from 1,301 outpatient evaluees who underwent testing with the Computerized Assessment of Response Bias (CARB), the Test of Memory Malingering (TOMM), and the Word Memory Test (WMT) in a forensic evaluation context. Under various data models, Markov chain Monte Carlo methods implemented via WinBUGS converged to target distributions that permitted Bayesian estimates of SVT accuracy. Under the most plausible model (conditional dependence in test results), classification accuracies (expressed as area under the "trapezoidal" receiver operating characteristic curve ± standard deviation) were as follows: CARB = 0.765 ± 0.030, WMT = 0.929 ± 0.020, and TOMM = 0.771 ± 0.034. At decision thresholds that hold false positive rates at 0.02, the SVTs would detect invalid responses (true positives) at rates of approximately 35%, 65%, and 49%, respectively, for the 3 tests. Though LCM methods have limitations, this study suggests that they offer an approach to SVT evaluation that avoids methodological pitfalls of known-group research designs while retaining ecological validity that is absent in simulation studies.

show abstract

Quantifying the accuracy of forensic examiners in the absence of a “gold standard”.

Cited by 21 publications

References 56 publications

Estimation of diagnostic test accuracy without full verification: a review of latent class methods

Estimation of diagnostic test accuracy without full verification: a review of latent class methods

Ten Year Research Update (2001–2010): Evaluations for Competence to Stand Trial (Adjudicative Competence)

Estimating the accuracy of neurocognitive effort measures in the absence of a “gold standard”.

Contact Info

Product

Resources

About