The Test Validation Summary

Frederick, Richard I.; Bowden, Stephen C.

doi:10.1177/1073191108325005

Cited by 33 publications

(46 citation statements)

References 54 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Following Frederick and Bowden (2009b), the two groups in each study were collapsed into a single mixed group, resulting in one group per study. Following Frederick and Bowden (2009b), the two groups in each study were collapsed into a single mixed group, resulting in one group per study.…”

Section: Methodsmentioning

confidence: 99%

Considerations underlying the use of mixed group validation.

Jewsbury¹,

Bowden²

2013

Psychological Assessment

Self Cite

View full text Add to dashboard Cite

Mixed Group Validation (MGV) is an approach for estimating the diagnostic accuracy of tests. MGV is a promising alternative to the more commonly used Known Groups Validation (KGV) approach for estimating diagnostic accuracy. The advantage of MGV lies in the fact that the approach does not require a perfect external validity criterion or gold standard. However, the research designs where MGV is most appropriate have not been thoroughly explored. We give a brief description of the ideal research design to minimize error for MGV studies, test whether the MGV assumptions hold with clinical data, evaluate whether there is evidence of assumption violation among published MGV studies, give a practical description of the MGV assumptions, and describe an example of an optimal use of MGV. Ultimately, we conclude that MGV is not generally superior to KGV but may be used in some cases where the assumptions and standard error have been considered appropriately.

show abstract

Section: Methodsmentioning

confidence: 99%

Considerations underlying the use of mixed group validation.

Jewsbury¹,

Bowden²

2013

Psychological Assessment

Self Cite

View full text Add to dashboard Cite

show abstract

“…In our second example, we illustrate the clinical utility of multiple-level LRs using WMA data from Loring et al ( in press ). It should be noted that the Wada data are unusual in that the prevalence of the two diagnostic categories of interest (left and right TLE) is approximately 50% and is higher than those observed for most other clinical conditions (e.g., Frederick & Bowden, 2008 ;Meehl & Rosen, 1955 ;Woods et al, 2003 ). However, the Wada data illustrates how multiplelevel LRs provides a valuable clinical technique to estimate posttest probability after obtaining any observed score.…”

Section: Discussionmentioning

confidence: 87%

The diagnostic utility of multiple-level likelihood ratios

Bowden

Loring

2009

J Int Neuropsychol Soc

Self Cite

View full text Add to dashboard Cite

Clinicians are accustomed to interpreting diagnostic test scores in terms of sensitivity and specificity. Many clinicians also appreciate that sensitivity and specificity need to be interpreted in terms of local base rates (i.e., pretest probability). However, most neuropsychological tests contain a wide range of scores. Important diagnostic information may be sacrificed when valid test scores are reduced to the simple dichotomy of "positive" or "negative" diagnosis that underlies sensitivity and specificity analysis. The purpose of this study is to provide an introduction to multiple-level likelihood ratios, a method for preserving the information in a wider range of scores. These statistics are first described using a hypothetical example of dementia screening, then with patient data from an epilepsy surgery sample. Multiple-level likelihood ratios have several advantages over sensitivity and specificity analysis because they are applied across a wider range of diagnostic scores, and generalize to settings with different base rates. We suggest that the diagnostic validity of many psychological tests may be underestimated by relying solely on traditional dichotomous sensitivity and specificity analysis.

show abstract

“…The Test Validation Summary (TVS; Frederick & Bowden, ) can assist clinicians who want to report predictive values in their risk assessments. The TVS is a graphing method that displays TPR, FPR, BR, PPV, NPV, and the proportion of positive scores in one figure for any particular cut‐off score on any particular test .…”

Section: The Test Validation Summarymentioning

confidence: 99%

Classification Accuracy of Actuarial Risk Assessment Instruments

Neller¹,

Frederick²

2013

Behavioral Sci & The Law

Self Cite

View full text Add to dashboard Cite

Users of commonly employed actuarial risk assessment instruments (ARAIs) hope to generate numerical probability statements about risk; however, ARAI manuals often do not explicitly report data that are essential for understanding the classification accuracy of the instruments. In addition, ARAI manuals often contain data that have the potential for misinterpretation. The authors of the present article address the accurate generation of probability statements. First, they illustrate how the reporting of numerical probability statements based on proportions rather than predictive values can mislead users of ARAIs. Next, they report essential test characteristics that, to date, have gone largely unreported in ARAI manuals. Then they discuss a graphing method that can enhance the practice of clinicians who communicate risk via numerical probability statements. After the authors review several strategies for selecting optimal cut-off scores, they show how the graphing method can be used to estimate positive predictive values for each cut-off score of commonly used ARAIs, across all possible base rates. They also show how the graphing method can be used to estimate base rates of violent recidivism in local samples.

show abstract

The Test Validation Summary

Cited by 33 publications

References 54 publications

Considerations underlying the use of mixed group validation.

Considerations underlying the use of mixed group validation.

The diagnostic utility of multiple-level likelihood ratios

Classification Accuracy of Actuarial Risk Assessment Instruments

Contact Info

Product

Resources

About