Meta‐analysis of diagnostic accuracy studies with multiple thresholds: Comparison of different approaches

Zapf, Antonia; Albert, Christian; Frömke, Cornelia; Haase, Michael; Hoyer, Annika; Jones, Hayley E; Rücker, Gerta

doi:10.1002/bimj.202000091

Cited by 9 publications

(16 citation statements)

References 34 publications

(57 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Although statistically sound, the methods are dependent on a single pair of specificity and/or sensitivity from each study that must be selected at a particular threshold and point, resulting in most of the data being omitted. 24,25,36 Further, the models used are unable to provide optimal concentration thresholds. 24,25,36 Pooled sensitivities and specificities provide clinicians with no information regarding which threshold or point is preferred, limiting their clinical utility.…”

Section: Discussionmentioning

confidence: 99%

“…24,25,36 Further, the models used are unable to provide optimal concentration thresholds. 24,25,36 Pooled sensitivities and specificities provide clinicians with no information regarding which threshold or point is preferred, limiting their clinical utility. The results of the present study indicate that both NSE and S100β had lower pooled AUC and specificity.…”

Section: Discussionmentioning

confidence: 99%

“…This is not surprising because the analysis by Wang et al 30 used a single pair of specificity and/or sensitivity with the greatest specificity for each study, which may bias the result toward greater specificity. 24,25,36 Our analysis used all reported values of specificity and sensitivity stratified by time. The discrepancy in pooled AUC and specificity may also be attributable to additional studies included in our review published since 2019 and the inclusion of studies that evaluated neurologic outcome up to 6 months following cardiac arrest.…”

Section: Discussionmentioning

confidence: 99%

“…Instead, we used a 2-stage random-effects model (including a linear mixed-effects model) to allow for the integration of multiple thresholds within each study and for each individual biomarker (diagmeta package in R statistical software [R Foundation for Statistical Computing]). 24,25 Using this model, we were able to calculate a summary receiver operating characteristic (SROC) curve for each biomarker at 48 hours following cardiac arrest as our primary outcome. We chose to use 48 hours because neurologic prognostication typically occurs at least 24 hours after cardiac arrest, with different time recommendations for patients treated with targeted temperature management (TTM) vs those kept normothermic.…”

Section: Key Pointsmentioning

confidence: 99%

“…Heterogeneity was assessed by clinical evaluation of each included study and by visual inspection of individual study ROC plots. 25 Median concentration and spread (IQR or range) and/or mean (SD) for each biomarker at 0 hours, 24 hours, 48 hours, and 72 hours were extracted from available studies. We performed a meta-analysis of median concentration for each biomarker at each of the selected times using the quantile estimation method (metamedian package in R statistical software).…”

Section: Key Pointsmentioning

confidence: 99%

See 4 more Smart Citations

Neurologic Prognostication After Cardiac Arrest Using Brain Biomarkers

et al. 2022

View full text Add to dashboard Cite

IMPORTANCE Brain injury biomarkers released into circulation from the injured neurovascular unit are important prognostic tools in patients with cardiac arrest who develop hypoxic ischemic brain injury (HIBI) after return of spontaneous circulation (ROSC).OBJECTIVE To assess the neuroprognostic utility of bloodborne brain injury biomarkers in patients with cardiac arrest with HIBI.DATA SOURCES Studies in electronic databases from inception to September 15, 2021. These databases included MEDLINE, Embase, Evidence-Based Medicine Reviews, CINAHL, Cochrane Database of Systematic Reviews, and the World Health Organization Global Health Library.STUDY SELECTION Articles included in this systmatic review and meta-analysis were independently assessed by 2 reviewers. We included studies that investigated neuron-specific enolase, S100 calcium-binding protein β, glial fibrillary acidic protein, neurofilament light, tau, or ubiquitin carboxyl hydrolase L1 in patients with cardiac arrest aged 18 years and older for neurologic prognostication. We excluded studies that did not (1) dichotomize neurologic outcome as favorable vs unfavorable, (2) specify the timing of blood sampling or outcome determination, or (3) report diagnostic test accuracy or biomarker concentration.DATA EXTRACTION AND SYNTHESIS Data on the study design, inclusion and exclusion criteria, brain biomarkers levels, diagnostic test accuracy, and neurologic outcome were recorded. This study was conducted in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guideline.MAIN OUTCOMES AND MEASURES Summary receiver operating characteristic curve analysis was used to calculate the area under the curve, sensitivity, specificity, and optimal thresholds for each biomarker. Risk of bias and concerns of applicability were assessed with the Quality Assessment of Diagnostic Accuracy Studies (QUADAS-2) tool. RESULTSWe identified 2953 studies, of which 86 studies with 10 567 patients (7777 men [73.6] and 2790 women [26.4]; pooled mean [SD] age, 62.8 [10.2] years) were included. Biomarker analysis at 48 hours after ROSC demonstrated that neurofilament light had the highest predictive value for unfavorable neurologic outcome, with an area under the curve of 0.92 (95% CI, 0.84-0.97). Subgroup analyses of patients treated with targeted temperature management and those who specifically had an out-of-hospital cardiac arrest showed similar results (targeted temperature management, 0.92 [95% CI, 0.86-0.95] and out-of-hospital cardiac arrest, 0.93 [95% CI, 0.86-0.97]).CONCLUSIONS AND RELEVANCE Neurofilament light, which reflects white matter damage and axonal injury, yielded the highest accuracy in predicting neurologic outcome in patients with HIBI at 48 hours after ROSC.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Discussionmentioning

confidence: 99%

Section: Discussionmentioning

confidence: 99%

Section: Key Pointsmentioning

confidence: 99%

Section: Key Pointsmentioning

confidence: 99%

See 3 more Smart Citations

Neurologic Prognostication After Cardiac Arrest Using Brain Biomarkers

et al. 2022

View full text Add to dashboard Cite

show abstract

Undertaking meta‐analysis

Takwoingi¹,

Dendukuri²,

Schiller³

et al. 2023

Cochrane Handbook for Systematic Reviews of Diagnostic Test Accuracy

View full text Add to dashboard Cite

Explaining the optimistic performance evaluation of newly proposed methods: A cross‐design validation experiment

et al. 2023

View full text Add to dashboard Cite

The constant development of new data analysis methods in many fields of research is accompanied by an increasing awareness that these new methods often perform better in their introductory paper than in subsequent comparison studies conducted by other researchers. We attempt to explain this discrepancy by conducting a systematic experiment that we call “cross‐design validation of methods”. In the experiment, we select two methods designed for the same data analysis task, reproduce the results shown in each paper, and then reevaluate each method based on the study design (i.e., datasets, competing methods, and evaluation criteria) that was used to show the abilities of the other method. We conduct the experiment for two data analysis tasks, namely cancer subtyping using multiomic data and differential gene expression analysis. Three of the four methods included in the experiment indeed perform worse when they are evaluated on the new study design, which is mainly caused by the different datasets. Apart from illustrating the many degrees of freedom existing in the assessment of a method and their effect on its performance, our experiment suggests that the performance discrepancies between original and subsequent papers may not only be caused by the nonneutrality of the authors proposing the new method but also by differences regarding the level of expertise and field of application. Authors of new methods should thus focus not only on a transparent and extensive evaluation but also on comprehensive method documentation that enables the correct use of their methods in subsequent studies.

show abstract

Meta‐analysis of diagnostic accuracy studies with multiple thresholds: Comparison of different approaches

Cited by 9 publications

References 34 publications

Neurologic Prognostication After Cardiac Arrest Using Brain Biomarkers

Neurologic Prognostication After Cardiac Arrest Using Brain Biomarkers

Undertaking meta‐analysis

Explaining the optimistic performance evaluation of newly proposed methods: A cross‐design validation experiment

Contact Info

Product

Resources

About