Understanding Sources of Bias in Diagnostic Accuracy Studies

Schmidt, Robert L.; Factor, Rachel E.

doi:10.5858/arpa.2012-0198-ra

Cited by 140 publications

(127 citation statements)

References 19 publications

(16 reference statements)

Supporting

Mentioning

125

Contrasting

Unclassified

Order By: Relevance

“…The study participants must be representative of the study entrants in order for the study participants. Therefore, a study ideally should enroll a consecutive or random sample of eligible patients with suspected disease to prevent the potential patient selection bias [39], [50].…”

Section: Discussionmentioning

confidence: 99%

Diagnostic Accuracy of Ber-EP4 for Metastatic Adenocarcinoma in Serous Effusions: A Meta-Analysis

Wang

et al. 2014

PLoS ONE

View full text Add to dashboard Cite

Numerous studies have investigated the utility of Ber-EP4 in differentiating metastatic adenocarcinoma (MAC) from malignant epithelial mesothelioma (MM) and/or reactive mesothelial cells (RM) in serous effusions. However, the results remain controversial. The aim of this study is to determine the overall accuracy of Ber-EP4 in serous effusions for MAC through a meta-analysis of published studies. Publications addressing the accuracy of Ber-EP4 in the diagnosis of MAC were selected from the Pubmed, Embase and Cochrane Library. Data from selected studies were pooled to yield summary sensitivity, specificity, positive and negative likelihood ratio (LR), diagnostic odds ratio (DOR), and receiver operating characteristic (SROC) curve. Statistical analysis was performed by Meta-Disc 1.4 and STATA 12.0 softwares. 29 studies, based on 2646 patients, met the inclusion criteria and the summary estimating for Ber-EP4 in the diagnosis of MAC were: sensitivity 0.8 (95% CI: 0.78–0.82), specificity 0.94 (95% CI: 0.93–0.96), positive likelihood ratio (PLR) 12.72 (95% CI: 8.66–18.7), negative likelihood ratio (NLR) 0.18 (95% CI: 0.12–0.26) and diagnostic odds ratio 95.05 (95% CI: 57.26–157.77). The SROC curve indicated that the maximum joint sensitivity and specificity (Q-value) was 0.91; the area under the curve was 0.96. Our findings suggest that BER-EP4 may be a useful diagnostic adjunctive tool for confirming MAC in serous effusions.

show abstract

Section: Discussionmentioning

confidence: 99%

Diagnostic Accuracy of Ber-EP4 for Metastatic Adenocarcinoma in Serous Effusions: A Meta-Analysis

Wang

et al. 2014

PLoS ONE

View full text Add to dashboard Cite

show abstract

“…However, DTA studies are subject to unique sources of bias that can distort estimates of a method's sensitivity and specificity. 1 Bias occurs when there is a systematic difference between the true value and the observed value. Partial verification bias and classification bias are types of bias that commonly occur in DTA studies for fine-needle aspiration biopsy (FNAB).…”

Section: Introductionmentioning

confidence: 99%

Verification and classification bias interactions in diagnostic test accuracy studies for fine‐needle aspiration biopsy

Schmidt

Walker

Cohen

2014

Cancer Cytopathology

Self Cite

View full text Add to dashboard Cite

BACKGROUND: Reliable estimates of accuracy are important for any diagnostic test. Diagnostic accuracy studies are subject to unique sources of bias. Verification bias and classification bias are 2 sources of bias that commonly occur in diagnostic accuracy studies. Statistical methods are available to estimate the impact of these sources of bias when they occur alone. The impact of interactions when these types of bias occur together has not been investigated. METHODS: We developed mathematical relationships to show the combined effect of verification bias and classification bias. A wide range of case scenarios were generated to assess the impact of bias components and interactions on total bias. RESULTS: Interactions between verification bias and classification bias caused overestimation of sensitivity and underestimation of specificity. Interactions had more effect on sensitivity than specificity. Sensitivity was overestimated by at least 7% in approximately 6% of the tested scenarios. Specificity was underestimated by at least 7% in less than 0.1% of the scenarios. CONCLUSIONS:Interactions between verification bias and classification bias create distortions in accuracy estimates that are greater than would be predicted from each source of bias acting independently. Cancer (Cancer Cytopathol) 2015;123:193

show abstract

“…This statistically derived probability calculator was created to enhance decision making when interpreting difficult breast FNA and to reduce sources of bias (29,30).The methodology used for this project was intended to reduce subjectivity by providing an evidence based tool which could calculate the risk of malignancy of the combined predictive criteria thereby assist in allocating a lesion into a suitable diagnostic category. However, this calculator has limitations, as it focuses only on the microscopic features and does not include the clinical and imaging findings of the triple test.…”

Section: Resultsmentioning

confidence: 99%

Stratification of risk of malignancy in atypical breast fine needle aspiration: a cytomorphological approach

Weigner

Zardawi²,

Braye

et al. 2017

J Diagnostic Path

View full text Add to dashboard Cite

SummaryAlthough atypical (C3) breast cytology is a legitimate reporting category, it is not very useful clinically because it yields benign as well as malignant histological outcomes. Therefore, a cytomorphological approach to minimise inappropriate use of the C3 category may improve the clinical usefulness of this category. In a previous training set of 180 atypical breast cytology (C3) cases we had identified the best discriminating cytological criteria to predict the histological outcomes (malignant, benign proliferative and benign non-proliferative). Using a selection of these previously identified statistically significant criteria (cystic background, cohesiveness, myoepithelial cells or bare bipolar nuclei, papillary fragments and tubules) we tested their ability to predict the same outcomes on 182 subsequent C3 cases (validation set). The diagnostic accuracy for each outcome was compared with the training set. A probability calculator with nominated cut-points was developed to stratify and re-grade C3 cases. Cases outside the cut-points were either up-graded to suspicious (C4) or down-graded to benign (C2). Re-graded cases were reviewed to identify reasons for over or under-interpretation. Statistical analysis showed comparable diagnostic accuracy between the training set and the validation set for malignant and benign proliferative outcomes but not for the benign non-proliferative outcome. The probability calculator resulted in an up-grade of 18% (33/182) ofthe cases. Malignant histology was seen in 25/33 (76%) of the up-graded cases but 8/33 (24%) produced a proliferative non-malignant outcome. A meaningful lower cut-point could not be established without including malignant cases. The C3 category remains a legitimate with a heterogeneous mix of pathological entities. Attempts to minimise inappropriate allocation of C3 cases into this category have met with limited success.

show abstract

Understanding Sources of Bias in Diagnostic Accuracy Studies

Abstract: There are several sources of bias that are unique to diagnostic accuracy studies. Because pathologists are both consumers and producers of such studies, it is important that they be aware of the risk of bias.

Cited by 140 publications

References 19 publications

Diagnostic Accuracy of Ber-EP4 for Metastatic Adenocarcinoma in Serous Effusions: A Meta-Analysis

Diagnostic Accuracy of Ber-EP4 for Metastatic Adenocarcinoma in Serous Effusions: A Meta-Analysis

Verification and classification bias interactions in diagnostic test accuracy studies for fine‐needle aspiration biopsy

Stratification of risk of malignancy in atypical breast fine needle aspiration: a cytomorphological approach

Contact Info

Product

Resources

About