Testing for publication bias in diagnostic meta‐analysis: a simulation study

Abstract

The present study investigates the performance of several statistical tests to detect publication bias in diagnostic meta-analysis by means of simulation. While bivariate models should be used to pool data from primary studies in diagnostic meta-analysis, univariate measures of diagnostic accuracy are preferable for the purpose of detecting publication bias. In contrast to earlier research, which focused solely on the diagnostic odds ratio or its logarithm (ln !), the tests are combined with four different univariate measures of diagnostic accuracy. For each combination of test and univariate measure, both type I error rate and statistical power are examined under diverse conditions. The results indicate that tests based on linear regression or rank correlation cannot be recommended in diagnostic meta-analysis, because type I error rates are either inflated or power is too low, irrespective of the applied univariate measure. In contrast, the combination of trim and fill and ln ! has non-inflated or only slightly inflated type I error rates and medium to high power, even under extreme circumstances (at least when the number of studies per meta-analysis is large enough). Therefore, we recommend the application of trim and fill combined with ln ! to detect funnel plot asymmetry in diagnostic meta-analysis.research. Besides, the possibly most prominent reason is the non-significance of the results [5]. However, this non-significance rarely applies to diagnostic studies. Usually, findings on test accuracy contain Sen and Spe (or univariate measures (UMs) computed on their basis) together with 95% confidence intervals, but they do not state a null hypothesis [6,7]. Hence, in most cases, there is no statistical test that can potentially fail to be significant. Therefore, one might argue that PB is not a problem in diagnostic meta-analysis. Nevertheless, some approaches such as non-inferiority designs (in which a new treatment or diagnostic test is compared with an already established one) are based on significance tests even in diagnostic contexts [8][9][10]. In addition, as studies of diagnostic accuracy are often conducted as part of routine clinical practice, they may be abandoned more easily, which should increase the probability of PB [3,11]. Another reason accounting for PB in all types of meta-analysis is that dramatic findings are more likely to occur in primary studies and meta-analyses that are methodically weaker [12]. As scientific journals may prefer such findings, it is plausible that meta-analyses not only tend to overestimate effects because of unpublished studies, but that researchers also base their results on less elaborate literature search. This assumption has been supported by findings of Song et al. [11] who showed that publication bias was increased in diagnostic meta-analyses in which only few databases were used to find relevant studies. In addition, we share the view of one reviewer of this paper who suspects that potential mechanisms of PB in the diagnostic context will depend on the type of ...

Is bilingualism losing its advantage? A bibliometric approach

“…There are other useful methodologies more suitable to shed light to some of the issues affecting this field ( e . g ., the presence of a decline effect), such as meta-analysis [43]. Meta-analysis enables synthesizing the accumulated evidence in a research field, estimating the strength of effects and the variance in the distribution of effect-sizes across the studies analyzed [44].…”

Hepatitis C Core Antigen Testing for Diagnosis of Hepatitis C Virus Infection

“…We employed standard methods for systematic reviews and meta-analyses of diagnostic tests (14–18), including preparation of an a priori protocol (see Data Supplement) for the literature search, article selection, data extraction, quality assessment and analysis.…”

“…We generated funnel plots displaying the Youden Index (YI = Sensitivity + Specificity −1) versus the Standard Error (SE) for each study (18, 23). We additionally performed the trim and fill statistical assessment in STATA using the ‘metatrim’ command (24).…”

The Association between Age-Related Macular Degeneration and the Risk of Mortality

“…The RR were pooled using the random-effects DerSimonian and Laird models [22]. The possibility of publication bias was evaluated using the visual inspection of a funnel plot [23]. Moreover, subgroup analyses were conducted to evaluate the influences of the selected study and participant characteristics (including regions, case numbers, and types of AMD and number of prescriptions and follow-up year) on the results.…”