Abstract analysis method facilitates filtering low-methodological quality and high-bias risk systematic reviews on psoriasis interventions

Abstract

BackgroundArticle summaries’ information and structure may influence researchers/clinicians’ decisions to conduct deeper full-text analyses. Specifically, abstracts of systematic reviews (SRs) and meta-analyses (MA) should provide structured summaries for quick assessment. This study explored a method for determining the methodological quality and bias risk of full-text reviews using abstract information alone.MethodsSystematic literature searches for SRs and/or MA about psoriasis were undertaken on MEDLINE, EMBASE, and Cochrane database. For each review, quality, abstract-reporting completeness, full-text methodological quality, and bias risk were evaluated using Preferred Reporting Items for Systematic Reviews and Meta-analyses for abstracts (PRISMA-A), Assessing the Methodological Quality of Systematic Reviews (AMSTAR), and ROBIS tools, respectively. Article-, author-, and journal-derived metadata were systematically extracted from eligible studies using a piloted template, and explanatory variables concerning abstract-reporting quality were assessed using univariate and multivariate-regression models. Two classification models concerning SRs’ methodological quality and bias risk were developed based on per-item and total PRISMA-A scores and decision-tree algorithms. This work was supported, in part, by project ICI1400136 (JR). No funding was received from any pharmaceutical company.ResultsThis study analysed 139 SRs on psoriasis interventions. On average, they featured 56.7% of PRISMA-A items. The mean total PRISMA-A score was significantly higher for high-methodological-quality SRs than for moderate- and low-methodological-quality reviews. SRs with low-bias risk showed higher total PRISMA-A values than reviews with high-bias risk. In the final model, only ’authors per review > 6’ (OR: 1.098; 95%CI: 1.012-1.194), ’academic source of funding’ (OR: 3.630; 95%CI: 1.788-7.542), and ’PRISMA-endorsed journal’ (OR: 4.370; 95%CI: 1.785-10.98) predicted PRISMA-A variability. Reviews with a total PRISMA-A score < 6, lacking identification as SR or MA in the title, and lacking explanation concerning bias risk assessment methods were classified as low-methodological quality. Abstracts with a total PRISMA-A score ≥ 9, including main outcomes results and explanation bias risk assessment method were classified as having low-bias risk.ConclusionsThe methodological quality and bias risk of SRs may be determined by abstract’s quality and completeness analyses. Our proposal aimed to facilitate synthesis of evidence evaluation by clinical professionals lacking methodological skills. External validation is necessary.Electronic supplementary materialThe online version of this article (doi:10.1186/s12874-017-0460-z) contains supplementary material, which is available to authorized users.

Evaluation of Spin in the Abstracts of Systematic Reviews and Meta-Analyses Focused on the Treatment of Acne Vulgaris: Cross-Sectional Analysis

“…Methodologists have performed interesting work to aid in establishing the confidence of the results and conclusions of systematic reviews; however, more work is needed regarding systematic review abstracts. A study performed by Gómez-García et al [35] found that, on average, only 57% of the PRISMA-A items were included in the abstracts of systematic reviews and meta-analyses covering psoriasis treatments. They reported that studies with low-risk bias and high methodological quality had significantly more PRISMA-A items reported compared with studies with high-risk bias and low methodological quality.…”

Do not make clinical decisions based on abstracts of healthcare research: A systematic review

“…Gómez-García. 2017 [36] To assess abstracts reporting quality and to determine if the PRISMA-A indirectly captures the methodological quality and the risk of bias in the full texts.…”

“…endorsement, [26 , 36 , 47] positive outcomes statistical significance, [5 , 11] higher methodological quality, [26 , 27 , 36] less spin in the abstracts, [26] European countries, [41] and academic funding source. [36] On the other hand, some studies have found different results for the same variables, showing no association of abstracts' reporting quality with: publication year, [26 , 35 , 39] abstract word count, [32 , 36 , 39 , 41] journal impact factor, [5 , 36 , 41] number of centers, [26 , 35 , 36 , 38 , 39] number of authors, [3 , 32 , 35 , 38 , 39 , 41 , 46 , 49] abstract format, [26 , 27 , 29 , 35 , 36 , 39 , 41] reporting guidelines endorsement, [3 , 5 , 11 , 29 , 41] outcomes statistical significance, [35 , 38-40 , 43 , 46 , 47] methodological quality, [27] continent of first author [5 , 26 , 35 , 38-40 , 46] and funding source. [5 , 11 , 39 , 47] The most common items not adequately reported (less than 50% of abstracts reporting) from the CONSORT-A were: randomization, blinding, funding, numbers analyzed, harms, authors, trial design, participants, primary outcomes results, registration, number randomized, clearly defined primary outcomes, recruitment status, conclusions, interventions and objectives.…”

Priorities for Social and Humanities Projects Based on Text Analysis①

“…For some domains, this information may not be enough to achieve high performance in text mining tasks (Gonçalves et al, 2018). Also, the bias risk caused by abstract's quality and completeness exists (Gómez-García et al, 2017). The full text analysis has shown that substantial differences in profile and orientation can occur within the categories, such as methodological or empirical research (Glenisson et al, 2005).…”