Understanding the Direction of Bias in Studies of Diagnostic Test Accuracy

Abstract

Ordering and interpreting diagnostic tests is a critical part of emergency medicine (EM). In evaluating a study of diagnostic test accuracy, emergency physicians (EPs) need to recognize whether the study uses case-control or cross-sectional sampling and account for common biases. The authors group biases in studies of test accuracy into five categories: incorporation bias, partial verification bias, differential verification bias, imperfect gold standard bias, and spectrum bias. Other named biases are either equivalent to these biases or subtypes within these broader categories. The authors go beyond identifying a bias and predict the direction of its effect on sensitivity and specificity, providing numerical examples from published test accuracy studies. Understanding the direction of a bias may permit useful inferences from even a flawed study of test accuracy.

Critical review of SPECT imaging in pulmonary embolism

“…However, despite continued research and publication in this area over the past 5 years, it has not been proven that these expectations are correct. The sensitivity and specificity of SPECT determined from investigations that used planar scintigraphy or SPECT itself as a component of the reference standard are most likely overestimated as a result of incorporation bias [18,19]. Incorporation bias was present in the design of 80 % of the investigations that we reviewed.…”

“…We addressed the following topics: (1) V/Q SPECT using multidetector CT angiography as the reference standard, (2) V/Q SPECT using composite reference standards, (3) V/Q SPECT compared with planar V/Q scintigraphy, (4) V/Q SPECT/CT accuracy, (5) V/Q SPECT/ CT compared with multidetector CT angiography, (6) perfusion SPECT/CT plus planar ventilation scintigraphy compared with dual-energy CT imaging, (7) perfusion SPECT accuracy, (8) perfusion SPECT/CT compared with V/Q SPECT/CT, (9) perfusion SPECT/CT compared with planar V/Q scintigraphy, (10) outcome after SPECT diagnosis, (11) V/Q SPECT for follow-up after acute PE, (12) SPECT after administration of 99m Tc-labeled anti D-dimer monoclonal Fab' fragments or 99m Tc-apcitide. It is difficult to draw firm conclusions about the diagnostic accuracy of SPECT largely because of design-related biases [18,19]. The intuitive conclusions would be that SPECT enhances sensitivity for detecting perfusion defects, facilitates evaluation of regional V/Q matching or mismatching, and is further aided by combination with low-dose CT imaging.…”

“…Incorporation bias was present in the design of 80 % of the investigations that we reviewed. On the other hand, sensitivity is likely to be overestimated and specificity to be reduced in the presence of partial verification bias, defined as a design in which the gold standard test (CT angiography or conventional angiography) is more likely to be obtained if the test under examination (SPECT or planar imaging) is abnormal, and only a small portion of patients with a negative test under examination are verified by the gold standard test [18,19]. Partial verification bias was present in 15 % of the studies we reviewed.…”

“…Partial verification bias was present in 15 % of the studies we reviewed. Other biases [18] also occurred, including differential verification bias (15 % of studies), imperfect gold standard (40 % of studies), disease spectrum bias (30 % of studies), and test result bias (35 % of studies). In calculating the proportion of investigations with design bias, we included two substudies as separate investigations, because their methods differed from those used in their primary investigations.…”

“…When comparing individual lung segments, the sensitivity of SPECT was 56 out of 62 (90 %) and the specificity 75 out of 118 (64 %) ( Table 1) V/Q SPECT using composite reference standards Most investigations of the sensitivity and specificity of V/Q SPECT included the index test (SPECT) or planar V/Q results in the composite reference standard [21,22,[24][25][26][27][28][29][30][31][32][33][34][35][36] ( Table 1). This would introduce incorporation bias [18,37], which can be expected to result in an overestimation of the diagnostic accuracy [18,38].…”