Stepwise-Hierarchical Pooled Analysis for Synergistic Interpretation of Meta-analyses Involving Randomized and Observational Studies: Methodology Development

Abstract

Background The necessity of including observational studies in meta-analyses has been discussed in the literature, but a synergistic analysis method for combining randomized and observational studies has not been reported. Observational studies differ in validity depending on the degree of the confounders’ influence. Combining interpretations may be challenging, especially if the statistical directions are similar but the magnitude of the pooled results are different between randomized and observational studies (the ”gray zone”). Objective To overcome these hindrances, in this study, we aim to introduce a logical method for clinical interpretation of randomized and observational studies. Methods We designed a stepwise-hierarchical pooled analysis method to analyze both distribution trends and individual pooled results by dividing the included studies into at least three stages (eg, all studies, balanced studies, and randomized studies). Results According to the model, the validity of a hypothesis is mostly based on the pooled results of randomized studies (the highest stage). Ascending patterns in which effect size and statistical significance increase gradually with stage strengthen the validity of the hypothesis; in this case, the effect size of the observational studies is lower than that of the true effect (eg, because of the uncontrolled effect of negative confounders). Descending patterns in which decreasing effect size and statistical significance gradually weaken the validity of the hypothesis suggest that the effect size and statistical significance of the observational studies is larger than the true effect (eg, because of researchers’ bias). Conclusions We recommend using the stepwise-hierarchical pooled analysis approach for meta-analyses involving randomized and observational studies.

Comparison of radiofrequency ablation and ablative external radiotherapy for the treatment of intrahepatic malignancies: A hybrid meta-analysis

“…Although not as rigid as the pooled analysis of randomised controlled trials, the pooled results of studies with reliable comparability have very low heterogeneity and are consistently valid, increasing the reliability of hypothesis testing. 15 Although both RFA and ablative RT confer a high LC probability for small intrahepatic malignancies, RFA might yield suboptimal LC for tumours near major vessels or the diaphragm, or those exceeding 2–3 cm in size. 9 , [45] , [46] , [47] However, ablative RT is less limited by tumour location because it does not cause mechanical damage and can provide a prescribed dose within a relatively wide range.…”

“…The stepwise-hierarchical pooled analysis by Shin et al. 15 was referred to for the stepwise analysis methods and interpretation of the subject owing to scarce randomised literature. Subgroup analyses were also performed according to the disease, including HCC and colorectal cancer (CRC) with liver metastases.…”

Role of local ablative treatment in oligometastatic non-small cell lung cancer: a meta-analysis

“…We performed sensitivity analyses including randomized studies only, as well as pooled analyses including all studies that met the inclusion criteria. For the integrated interpretation of the pooled results of the overall and sensitivity analyses, we referred to the stepwise hierarchical pooled analysis for synergistic interpretation by Shin and Rim 18 (i.e. the ascending pattern of effect size, statistical significance, and validity of subgroups strengthens the hypothesis, whereas the descending pattern weakens).…”

Role of Local Treatment for Oligometastasis: A Comparability Based Meta-Analysis

“…Since the included studies dealt with heterogenous primary sites, subgroup analyses per primary were also subsequently performed. Subgroup analyses were performed according to hierarchical comparability and study designs, as suggested by Shin and Rim [ 14 ].…”