Quality assessment of reporting of randomization, allocation concealment, and blinding in traditional chinese medicine RCTs: A review of 3159 RCTs identified from 260 systematic reviews

Abstract

BackgroundRandomized controlled trials (RCTs) which are of poor quality tend to exaggerate the effect estimate and lead to wrong or misleading conclusions. The aim of this study is to assess the quality of randomization methods, allocation concealment and blinding within traditional Chinese medicine (TCM) RCTs, discuss issues identified for current TCM RCTs, and provide suggestions for quality improvement.MethodsWe searched Chinese Biomedical Database (CBM, 1978 to July 31, 2009) and the Cochrane Library (Issue 2, 2009) to collect TCM systematic reviews and meta-analyses according to inclusion/exclusion criteria, from which RCTs could be identified. The quality assessment involved whether the randomization methods, allocation concealment and blinding were adequate or not based the study reported. Stratified analyses were conducted of different types of diseases published in different journals (both Chinese and foreign) using different interventions. SPSS 15.0 software was used for statistic analyses.ResultsA total of 3159 RCTs were included, of which 2580 were published in Chinese journals and 579 in foreign journals. There were 381 (12%) RCTs which used adequate randomization methods; 207 (7%) RCTs which used adequate allocation concealment and 601 (19%) which used adequate blinding; there were 130 (4%) RCTs which both used adequate randomization methods and allocation concealment; and there were only 100 (3%) RCTs which used adequate randomization methods, allocation concealment, as well as blinding. In the RCTs published in foreign journals, the adequate randomization methods, allocation concealment and blinding accounted for a relatively large proportion (25%, 26%, and 60%, respectively) and increased with years, while in the RCTs published in Chinese journals, only the adequate randomization methods improved over time. The quality of non-drug intervention (chiefly acupuncture) RCTs was higher than that of drug intervention RCTs. In drug intervention, the quality of listed drugs is higher than the others. The quality of all included RCTs of all types of diseases was generally poor and no studies that were large in size and of high quality were found.ConclusionThe quality of the current TCM RCTs as judged by their publications is generally poor, especially those published in Chinese journals. In future, researchers of TCM RCTs should attach more importance to experimental design and methodological quality, receive relevant training, and improve reporting quality using the Consolidated Standards of Reporting Trials (CONSORT) statement, so as to improve the quality of TCM clinical research and ensure truth and reliability of conclusions.

The Efficacy of Brucea javanica Oil Emulsion Injection as Adjunctive Therapy for Advanced Non‐Small‐Cell Lung Cancer: A Meta‐Analysis

“…Although the quality of reporting RCTs of TCMs has been enhanced in the past decade, more and more academics realize that the percentage of high quality reports remains low, exacerbating the ROB problem relevant to TCMs trials [39–41]. Researchers should take responsibility for making registration of clinical trials [42], paying more attention to experiment design and methodological quality [43], and receiving education to write high quality report, so as to increase the credibility of TCMs studies.…”

The quality of reporting of randomized controlled trials of electroacupuncture for stroke

“…For example, information on the method of random sequence generation, primary outcome, sample size calculation, randomization stated in title, allocation concealment, and adequate blinding was reported in 34, 53, 45, 33, 25, and 18% of 616 reports indexed in PubMed in 2006, respectively [84]. Especially, in RCTs of traditional Chinese medicine that include herbal medicine, acupuncture and other no medication therapies, reporting of the key methods used for adequate randomization methods, adequate allocation concealment, adequate blinding, both adequate randomization methods and allocation concealment used, and all three used was only 12, 7, 19, 4, and 3% of 2580 reports, respectively [8]. Thus, several guidelines have been recommended to help incomplete and inaccurate reporting.…”

“…However, a large body of evidence indicated that the quality of reporting of RCTs remains sub-optimal [7]. Researchers have accumulated and suggested that the RCTs which were of poor methodological quality tend to exaggerate the treatment effects and result in misleading in health care at all levels [8]. So far, two studies have already been conducted to evaluate the quality of reporting of RCTs on acupuncture for stroke.…”

Elemene, One Ingredient of a Chinese Herb, Against Malignant Tumors: A Literature-based Meta-analysis

“…Hewitt et al (55) reported that more than 40% of trials published in 2004 in Western medical journals either failed to use adequate randomization methods or failed to report the method for concealment of allocation. Furthermore, He et al (56) reported that the methodological quality of 579 RCTs published in 2009 in foreign journals was generally poor, 75% failed to use adequate randomization methods, and 74% failed to report adequate allocation concealment.…”