Screening test accuracy studies: how valid are our conclusions? Application to visual inspection methods for cervical screening

Abstract

While the basic concepts associated with screening are simple, studying the value of new tests requires a very strict methodology. This paper summarizes lessons learned regarding appropriate methodologies to assess the value of new screening approaches using visual inspection with acetic acid (VIA), a screening test for cervical pre-cancerous lesions, as an example. In addition to being convenient to, safe for and acceptable by target community members, a screening test should be reliable and have good test characteristics (i.e. be able to discriminate well between early disease and non disease). Test reliability assesses the degree to which repeated measurements of the test yields the same result. To ensure reproducibility of study findings, test reliability should be assessed before any evaluation of test accuracy. The accuracy of a test (specificity and sensitivity) is measured using cross-sectional studies with adequate sample size. Several basic features are necessary to ensure internal validity for such studies: (a) final disease status data should be obtained for all subjects, (b) all tests results must be determined independently of previous results, (c) the reference standard used to determine the disease status should be accurate, (d) the full "spectrum" of the disease should be included in the study. The study results should also have external validity to be applicable to other populations to which the test will be applied. All these consideration are exemplified by 17 very heterogeneous studies published to date assessing VIA test accuracy. The assessment of a new screening test is the first step in researching a new cancer prevention strategy. For this reason, this step should be carefully addressed through rigorous studies.

Performance characteristics of Pap test, VIA, VILI, HR-HPV testing, cervicography, and colposcopy in diagnosis of significant cervical pathology

“…Interestingly, however, when correlated to the biopsy results, Sure-Path high-grade cases had 100 % concordance with the histological diagnosis, while DNACitoliq did only 40 % and Pap smear in 68.1 % (data previously reported; see reference [3]). As previously reported [4], VIA and VILI as stand-alone tests in the LAMS setting did not show the same good performance indicators as previously reported in other settings [13][14][15][16][17][18]. In our cohort, both sensitivity and specificity of VIA and VILI were lower than those of conventional Pap smear, except when the "suggesting cancer" cutoff was used, when both VIA and VILI were almost 100 % specific ( Table 4).…”

“…In addition to using the diagnostic assays as stand-alone tests, combining a test with high sensitivity with one demonstrating high specificity seems an attractive option [16]. When this was done in the present setting (Table 5), VIA and conventional Pap test disclosed the best balance of sensitivity, specificity, and unnecessary referrals to colposcopy.…”

“…In our cohort, both sensitivity and specificity of VIA and VILI were lower than those of conventional Pap smear, except when the "suggesting cancer" cutoff was used, when both VIA and VILI were almost 100 % specific ( Table 4). Most of the published reports on VIA/ILI are from the same authors and based on data from India or Africa, i.e., in geographic areas with very high prevalence of CC and high-grade CIN, which certainly influences the performance of VIA and VILI [13][14][15][16][17][18][19].…”

Accuracy of visual inspection with acetic acid for cervical cancer screening

“…Second, the measures of test accuracy are expected to correlate with the thresholds used (the higher the threshold, the lower the sensitivity and the higher the specificity) [14]. A threshold effect was indeed observed for sensitivity (71% for a CIN 2 and 59% for a CIN 3 threshold), but not for specificity (91% for a CIN 2 and 86% for a CIN 3 threshold).…”

“…However, important potential pitfalls such as the absence of control for the existence of gynecologic symptoms, different thresholds, or different recipients of confirmatory diagnosis were not considered. These omissions may have biased previous findings [14]. The objective of the present, comprehensive meta-analysis was to provide an updated summary estimate of the accuracy of VIA testing in detecting histologically confirmed CIN 2 (as disease threshold) or more advanced cervical lesions.…”