Quantifying the Added Value of a Diagnostic Test or Marker

Moons, Karel G.M.; Groot, Joris A H de; Línnet, Kristían; Reitsma, Johannes B.; Bossuyt, Patrick M.

doi:10.1373/clinchem.2012.182550

Cited by 119 publications

(107 citation statements)

References 38 publications

Supporting

Mentioning

103

Contrasting

Unclassified

Order By: Relevance

“…Previous articles in this series (1)(2)(3) have explained how accuracy is defined by comparing the results of the test or marker under evaluation with those of the clinical reference standard in the same patients. The reference standard is the best method for establishing the presence or absence of disease or, more generally, the target disease or condition.…”

confidence: 99%

Beyond Diagnostic Accuracy: The Clinical Utility of Diagnostic Tests

et al. 2012

Self Cite

View full text Add to dashboard Cite

Like any other medical technology or intervention, diagnostic tests should be thoroughly evaluated before their introduction into daily practice. Increasingly, decision makers, physicians, and other users of diagnostic tests request more than simple measures of a test's analytical or technical performance and diagnostic accuracy; they would also like to see testing lead to health benefits. In this last article of our series, we introduce the notion of clinical utility, which expresses—preferably in a quantitative form—to what extent diagnostic testing improves health outcomes relative to the current best alternative, which could be some other form of testing or no testing at all. In most cases, diagnostic tests improve patient outcomes by providing information that can be used to identify patients who will benefit from helpful downstream management actions, such as effective treatment in individuals with positive test results and no treatment for those with negative results. We describe how comparative randomized clinical trials can be used to estimate clinical utility. We contrast the definition of clinical utility with that of the personal utility of tests and markers. We show how diagnostic accuracy can be linked to clinical utility through an appropriate definition of the target condition in diagnostic-accuracy studies.

show abstract

confidence: 99%

Beyond Diagnostic Accuracy: The Clinical Utility of Diagnostic Tests

et al. 2012

Self Cite

View full text Add to dashboard Cite

show abstract

“…Incremental accuracy from combining tests, relative to a single test, can be more informative: for example, (13 ). For continuous tests, dichotomization into test positives and negatives may not always be indicated.…”

mentioning

confidence: 99%

STARD 2015: An Updated List of Essential Items for Reporting Diagnostic Accuracy Studies

et al. 2015

Self Cite

View full text Add to dashboard Cite

Incomplete reporting has been identified as a major source of avoidable waste in biomedical research. Essential information is often not provided in study reports, impeding the identification, critical appraisal, and replication of studies. To improve the quality of reporting of diagnostic accuracy studies, the Standards for Reporting of Diagnostic Accuracy Studies (STARD) statement was developed. Here we present STARD 2015, an updated list of 30 essential items that should be included in every report of a diagnostic accuracy study. This update incorporates recent evidence about sources of bias and variability in diagnostic accuracy and is intended to facilitate the use of STARD. As such, STARD 2015 may help to improve completeness and transparency in reporting of diagnostic accuracy studies. © 2015 American Association for Clinical ChemistryAs researchers, we talk and write about our studies, not just because we are happy-or disappointed-with the findings, but also to allow others to appreciate the validity of our methods, to enable our colleagues to replicate what we did, and to disclose our findings to clinicians, other health care professionals, and decision-makers, all of whom rely on the results of strong research to guide their actions.Unfortunately, deficiencies in the reporting of research have been highlighted in several areas of clinical medicine (1 ). Essential elements of study methods are often poorly described and sometimes completely omitted, making both critical appraisal and replication difficult, if not impossible. Sometimes study results are selectively reported, and other times researchers cannot resist unwarranted optimism in interpretation of their findings (2-4 ). These practices limit the value of the research and any downstream products or activities, such as systematic reviews and clinical practice guidelines.Reports of studies of medical tests are no exception. A growing number of evaluations have identified deficiencies in the reporting of test accuracy studies (5 ). These are studies in which a test is evaluated against a clinical reference standard, or gold standard; the results are typically reported as estimates of the test's sensitivity and specificity, which express how good the test is in correctly identifying patients as having the target condition. Other accuracy statistics can be used as well, such as the area under the ROC curve or positive and negative predictive values.Despite their apparent simplicity, such studies are at risk of bias (6, 7 ). If not all patients undergoing testing are included in the final analysis, for example, or if only healthy controls are included, the estimates of test accuracy may not reflect the performance of the test in clinical

show abstract

“…Conversely, it should assign risk values close to 0 to the observations to the left side (i.e., if ν < 1 −θ). A well discriminating model should therefore result in a predictiveness curve with steep slope (see also Huang et al, 2007;Huang & Pepe, 2009a;Moons et al, 2012). In Figure 2, for example, the step function that represents the "optimal" predictiveness curve jumps from zero to one at 1 −θ = 0.7.…”

Section: Measures Of Discriminationmentioning

confidence: 99%

“…The focus of this paper is on the predictiveness curve, which was first proposed by Huang, Pepe & Feng (2007) and has since been used by many authors as a graphical tool to visualize and evaluate calibration and discriminatory power (Pepe et al, 2008b;Gu & Pepe, 2009;Huang & Pepe, 2009b;Pepe, Gu & Morris, 2010;Moons et al, 2012;Steyerberg et al, 2014). The predictiveness curve depicts the risk distribution of a marker (or a marker combination) and is formally defined as follows: Let F be the cumulative distribution function (cdf) of the marker Y, and let ν = F(Y) ∈ (0, 1) be the ν-th percentile of Y.…”

Section: Introductionmentioning

confidence: 99%

“…Based on this interpretation, Pepe et al (2008b) suggested investigating the performance of prediction models by comparing their predictiveness curves. This strategy was adopted by Pencina, D'Agostino & Vasan (2010) and Moons et al (2012), who assessed the predictive capacity of a new marker by comparing the resulting predictiveness curves with and without the new marker. In particu-lar, Pencina et al (2010) noticed that in their data the predictiveness curves with and without the new marker hardly differed.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

The Residual-Based Predictiveness Curve: A Visual Tool to Assess the Performance of Prediction Models

Casalicchio¹,

Bischl²,

Boulesteix

et al. 2015

Biometrics

View full text Add to dashboard Cite

It is agreed among biostatisticians that prediction models for binary outcomes should satisfy two essential criteria: First, a prediction model should have a high discriminatory power, implying that it is able to clearly separate cases from controls. Second, the model should be well calibrated, meaning that the predicted risks should closely agree with the relative frequencies observed in the data. The focus of this work is on the predictiveness curve, which has been proposed by Huang et al. (Biometrics 63, 2007) as a graphical tool to assess the aforementioned criteria. By conducting a detailed analysis of its properties, we review the role of the predictiveness curve in the performance assessment of biomedical prediction models. In particular, we demonstrate that marker comparisons should not be based solely on the predictiveness curve, as it is not possible to consistently visualize the added predictive value of a new marker by comparing the predictiveness curves obtained from competing models. Based on our analysis, we propose the "residual-based predictiveness curve" (RBP curve), which addresses the aforementionened issue and which extends the original method to settings where the evaluation of a prediction model on independent test data is of particular interest. Similar to the predictiveness curve, the RBP curve reflects both the calibration and the discriminatory power of a prediction model. In addition, the curve can be conveniently used to conduct valid performance checks and marker comparisons.

show abstract

Quantifying the Added Value of a Diagnostic Test or Marker

Cited by 119 publications

References 38 publications

Beyond Diagnostic Accuracy: The Clinical Utility of Diagnostic Tests

Beyond Diagnostic Accuracy: The Clinical Utility of Diagnostic Tests

STARD 2015: An Updated List of Essential Items for Reporting Diagnostic Accuracy Studies

The Residual-Based Predictiveness Curve: A Visual Tool to Assess the Performance of Prediction Models

Contact Info

Product

Resources

About