ROC Curves in Clinical Chemistry: Uses, Misuses, and Possible Solutions

Abstract

Background: ROC curves have become the standard for describing and comparing the accuracy of diagnostic tests. Not surprisingly, ROC curves are used often by clinical chemists. Our aims were to observe how the accuracy of clinical laboratory diagnostic tests is assessed, compared, and reported in the literature; to identify common problems with the use of ROC curves; and to offer some possible solutions. Methods: We reviewed every original work using ROC curves and published in Clinical Chemistry in 2001 or 2002. For each article we recorded phase of the research, prospective or retrospective design, sample size, presence/absence of confidence intervals (CIs), nature of the statistical analysis, and major analysis problems. Results: Of 58 articles, 31% were phase I (exploratory), 50% were phase II (challenge), and 19% were phase III (advanced) studies. The studies increased in sample size from phase I to III and showed a progression in the use of prospective designs. Most phase I studies were powered to assess diagnostic tests with ROC areas >0.70. Thirty-eight percent of studies failed to include CIs for diagnostic test accuracy or the CIs were constructed inappropriately. Thirty-three percent of studies provided insufficient analysis for comparing diagnostic tests. Other problems included dichotomization of the gold standard scale and inappropriate analysis of the equivalence of two diagnostic tests. Conclusion: We identify available software and make some suggestions for sample size determination, testing for equivalence in diagnostic accuracy, and alternatives to a dichotomous classification of a continuous-scale

Is the current BMI obesity classification appropriate for black and white postmenopausal women?

“…The confidence intervals show that the overall performance of BMI as a method for classifying obese women is significantly higher than the traditional null hypothesis standard of AUC ¼ 0.5. [18][19][20] For black women, highest classification accuracy for obesity (using the 35, 38, and 40 %Fat cutpoints) corresponded to BMI values of 28.3, 28.4, and 28.4 kg/m 2 , respectively, and for white women, highest classification accuracy was associated with BMI values of 26.4, 26.9, and 27.5 kg/m 2 . The higher BMI cutpoints for black women compared to white women support the findings of the regression analysis, that for a given BMI, white women have a higher %Fat than black women.…”

Clinical proteomics: A need to define the field and to begin to set adequate standards

“…In the past and sometimes also nowadays, poor monitoring of data, inadequate statistical design and analysis, and dubious consent and ethical approval procedures have seriously flawed the viability of data in clinical research. As an example, although sensitivity and specificity are often quoted, an important tool in the analysis of diagnostic markers with several advantages is the receiver operating characteristic curve [8,9] and yet this is absent from most clinical proteomic studies to date. Following or emulating GCP/GCLP standards in clinical proteomics will reduce or eliminate many of the current problems.…”

Assessment of FUS-200 Performance: Comparison with Quantitative Urine Culture Shows that Identifying the Right Cutoff for Optimizing Analyzer Performance Is Challenging

“…These numbers appear to be reasonable for detecting diagnostic test accuracy. For phase I studies, where the diagnostic test is expected to have a fair AUC (0.70), 45 patients with the disease in question and 45 healthy controls will be needed to prove a type error of 5% with a power of 95% (27). Moreover, Metz recommends 100 cases for reliable ROC analysis (28).…”