Extension of the Composite Quality Score (CQS) as an appraisal tool for prospective, controlled clinical therapy trials–A systematic review of meta-epidemiological evidence

Abstract

Aim To conduct a survey of current meta-epidemiological studies to identify additional trial design characteristics that may be associated with significant over- or underestimation of the treatment effect and to use such identified characteristics as a basis for the formulation of new CQS appraisal criteria. Materials and methods We retrieved eligible studies from two systematic reviews on this topic (latest search May 2015) and searched the databases PubMed and Embase for further studies from June 2015 –March 2022. All data were extracted by one author and verified by another. Sufficiently homogeneous estimates from single studies were pooled using random-effects meta-analysis. Trial design characteristics associated with statistically significant estimates from single datasets (which could not be pooled) and meta-analyses were used as a basis to formulate new or amend existing CQS criteria. Results A total of 38 meta-epidemiological studies were identified. From these, seven trial design characteristics associated with statistically significant over- or underestimation of the true therapeutic effect were found. Conclusion One new criterion concerning double-blinding was added to the CQS, and the original criteria for concealing the random allocation sequence and for minimum sample size were amended.

Allocation concealment appraisal of clinical therapy trials using the extended Composite Quality Score (CQS-2) – An empirically based update (Preprint)

“…This study is a partial update of ndings from our previous systematic review of meta-epidemiological studies concerning the CQS appraisal criterion for allocation concealment [5]. In this review, we established evidence from two meta-epidemiological studies [8, 9] as a basis for formulating Criterion II of the CQS-2.…”

“…Against this background, the Composite Quality Score (CQS) is under development as a trial appraisal tool that seems to be an alternative to the RoB 2, based on its epistemological rigour [4], empirical evidence base [5], high inter-rater reliability (Brennan-Prediger coe cient 1.00; 95% CI: 0.94-1.00 [6] and 0.95, 95% CI: 0.87-1.00 [7]) and its apparent ease of application without prior training [6]. Its latest version (CQS-2) includes four criteria related to the random allocation to treatment groups, concealment of such allocation, double-blinding and sample size minimum.…”

Allocation concealment appraisal of clinical therapy trials using the extended Composite Quality Score (CQS-2): An empirically based update (Protocol)

“…Criterion II of the CQS-2 was developed based on evidence from two meta-epidemiological studies [8,9] for the appraisal of allocation concealment in trials [5]. The evidence indicated a statistically significant larger effect estimate for trials with 'inadequate' or 'unclear' allocation concealment (dSMD 0.15; 95%CI: 0.03 to 0.28; I 2 = 0%) compared to trials where allocation concealment was judged to be 'adequate'.…”

“…The evidence indicated a statistically significant larger effect estimate for trials with 'inadequate' or 'unclear' allocation concealment (dSMD 0.15; 95%CI: 0.03 to 0.28; I 2 = 0%) compared to trials where allocation concealment was judged to be 'adequate'. The evidence from both studies combined the results of 379 clinical, dental, oral and craniofacial trials [5]. Adequate allocation concealment was specified as concealment of the random allocation sequence that included (verbatim): "centralized or pharmacy-controlled randomization; coded identical containers administered serially; onsite computer system combined with allocations kept in a locked unreadable computer file; sequentially numbered, sealed, opaque envelopes and similar schemes ensuring that patient and clinician were unaware of the allocation, along with reassurance that the person who generated the allocation scheme did not administer it" [9].…”

“…Against this background, the Composite Quality Score (CQS) is under development as a trial appraisal tool that seems to be an alternative, based on its epistemological rigor [4], empirical evidence base [5], high inter-rater reliability (Brennan-Prediger coefficient 1.00; 95% CI: 0.94 -1.00 [6] and 0.95, 95% CI: 0.87-1.00 [7]) and its apparent ease of application without prior training [6]. Its latest version (CQS-2) includes four criteria related to the random allocation to treatment groups, concealment of such allocation, double blinding and sample size minimum.…”