Methodologic considerations in comparing imaging methods

Abstract

Current methods for evaluating and comparing imaging methods may be inadequate David J. Ott1 in several important aspects. Prospective investigations often fail to provide uniform condftions for data collection due to variable physician skills in performing the studies being evaluated. The double-blind format, although seemingly objective, is inherently unable to prevent the effects of examiner or observer prejudice when imaging methods are being compared. Commonly used statistical terms are limited in their ability to characterize the clinical efficacy of imaging methods, and are easily misused. Reference examinations, or "gold standards," may be used in a manner preordaining an inferior result for the diagnostic method under evaluation. These problems are discussed and examples of their effects are presented. Suggestions are presented for minimizing existing methodologic limitations.

Pediatric hypertension: an approach to imaging

“…US has the advantage of being noninvasive, not subjecting the patient to radiation and being more diagnostic in patients with decreased renal function or extrarenal neoplasms. The one disadvantage of US relative to IVP seems to be its decreased sensitivity in the detection of renal scarring [23]. However, the only patient with this disorder in our study was detected by US.…”

Investigation of childhood blunt abdominal trauma: a practical approach using ultrasound as the initial diagnostic modality

“…The clinical outcome suggested no false negative examinations, but it is possible that minor clinically silent injuries to intraabdominal organs might have been missed by the various investigations. The false negative rate in this study is therefore difficult to assess and our results are expressed as percentage positive examinations rather than as true sensitivity [7].…”

“…It is difficult to establish a "gold standard" for use as a basis for comparison of various imaging methods in a clinical population which heals largely without surgery [7]. By necessity, the clinical outcome takes the place of surgery and autopsy.…”

Definition of terms used to judge the efficacy of diagnostic tests: a graphic approach.

“…Whereas the first two of these indexes are characteristics of the test itself and independent of the prevalence of the disease in the examined population, the latter two indexes were established to estimate the outcome of a test in a population and are dependent on the prevalence of the disease. These four indexes are widely accepted, and have been the subject of many excellent and highly readable articles [2][3][4][5]. Nonetheless, they are often misused in medical journals [6,7].…”

“…3E). However, this composite index has several serious drawbacks and should be avoided [3,4]. The diagram also may be used to demonstrate the effect of prevalence of disease on the four different indexes, or the effects of transferring a test from being used in a hospital population to being applied in a screening situation (Fig.…”