Methodology and reporting of meta-analyses in the neurosurgical literature

Abstract

Object. Neurosurgeons are inundated with vast amounts of new clinical research on a daily basis, making it difficult and time-consuming to keep up with the latest literature. Meta-analysis is an extension of a systematic review that employs statistical techniques to pool the data from the literature in order to calculate a cumulative effect size. This is done to answer a clearly defined a priori question. Despite their increasing popularity in the neurosurgery literature, meta-analyses have not been scrutinized in terms of reporting and methodology.Methods. The authors performed a literature search using PubMed/MEDLINE to locate all meta-analyses that have been published in the JNS Publishing Group journals (Journal of Neurosurgery, Journal of Neurosurgery: Pediatrics, Journal of Neurosurgery: Spine, and Neurosurgical Focus) or Neurosurgery. Accepted checklists for reporting (PRISMA) and methodology (AMSTAR) were applied to each meta-analysis, and the number of items within each checklist that were satisfactorily fulfilled was recorded. The authors sought to answer 4 specific questions: Are meta-analyses improving 1) with time; 2) when the study met their definition of a meta-analysis; 3) when clinicians collaborated with a potential expert in meta-analysis; and 4) when the meta-analysis was the only focus of the paper?Results. Seventy-two meta-analyses were published in the JNS Publishing Group journals and Neurosurgery between 1990 and 2012. The number of published meta-analyses has increased dramatically in the last several years. The most common topics were vascular, and most were based on observational studies. Only 11 papers were prepared using an established checklist. The average AMSTAR and PRISMA scores (proportion of items satisfactorily fulfilled divided by the total number of eligible items in the respective instrument) were 31% and 55%, respectively. Major deficiencies were identified, including the lack of a comprehensive search strategy, study selection and data extraction, assessment of heterogeneity, publication bias, and study quality. Almost one-third of the papers did not meet our basic definition of a meta-analysis. The quality of reporting and methodology was better 1) when the study met our definition of a meta-analysis; 2) when one or more of the authors had experience or expertise in conducting a meta-analysis; 3) when the meta-analysis was not conducted alongside an evaluation of the authors' own data; and 4) in more recent studies.Conclusions. Reporting and methodology of meta-analyses in the neurosurgery literature is excessively variable and overall poor. As these papers are being published with increasing frequency, neurosurgical journals need to adopt a clear definition of a meta-analysis and insist that they be created using checklists for both reporting and methodology. Standardization will ensure high-quality publications.

Methodological quality assessment of paper-based systematic reviews published in oral health

“…For example, the reporting of characteristics of the primary studies included in the paper-based SR was described in the majority of the SR in our sample. These results are in agreement with other systematic reviews from different medical fields such as pharmacology [19] and neurosurgery [20]. A probable explanation for this results is the flexibility of the AMSTAR checklist on accepting different forms for presenting the characteristics of included studies, i.e., table or text format.…”

“…Question 3 (a comprehensive literature search of more than one electronic database) received the second-most positive scores in the present assessment, reflecting similar results from SR published in dentistry [21] and other fields [22,23]. Nevertheless, some SR provided poor results for this criterion [6,20]. One possible explanation is the stricter guidelines for the interpretation of the question.…”

“…The three least common topics were restorative dentistry (n=15; 2.3 %), pain (n=11; 1.7 %), and other unclassified studies (n=25; 3.9 %). Although the median number of studies per SR was [15][16][17][18][19][20][21][22][23][24][25][26][27][28], the majority (70.2 %, n=450) included fewer than 24 studies in their review, while 2.0 % (n=13) included more than 100 studies. Most studies failed to describe study characteristics (88.2 %, n = 567), with terms such as Badults^(6.5 %, n = 42) or Bpaediatrics^(5.3 %, n = 34) populations used sparingly among the studies.…”

Case-control studies in neurosurgery

“…This is contrary to our recent study evaluating metaanalyses in the neurosurgical literature, in which we did find a strong positive correlation. 53 We offer 4 possible explanations: 1) our study was underpowered to detect a positive impact; 2) experts were not consistently acknowledged in true case-control studies; 3) acknowledged experts did not consistently participate in either the design or reporting of the study, but only performed statistical analysis; and 4) some individuals considered experts may not be able to distinguish between study designs. Several prominent epidemiologists have commented on the final point.…”

Cost-effectiveness analysis in minimally invasive spine surgery

“…Klimo et al discussed the methodology and utilization of the Primary Reporting Items for Systematic reviews and Meta-Analyses (PRIS-MA), A Measurement Tool to Assess Systematic Reviews (AMSTAR), and Meta-Analysis Of Observational Studies in Epidemiology (MOOSE) to identify high-quality reviews and encourage the development of high-quality reviews in future studies. 12 These tools allow for more objective evaluation of meta-analyses and can help authors in identifying the points necessary to make a high-quality study. Additionally, Consolidated Standards of Reporting Trials (CONSORT) has been used to assess RCTs and provide higher standards of reporting.…”