Quality, Evolution, and Clinical Implications of Randomized, Controlled Trials on the Treatment of Lung Cancer

Abstract

A review of 150 published randomized trials on the treatment of lung cancer showed serious methodological drawbacks. Handling of withdrawals (only 7 trials had no dropouts), a priori estimates of sample size (only 9 trials specified the required number of patients), blinding of randomization (only 22 trials had a satisfactory procedure), and information on eligible nonrandomized patients (only 13 studies reported it precisely) were areas of major concern. Although trial quality improved over time both in design/execution (study size estimation and analysis by prognostic factors became more frequent) and reporting (information on patients' characteristics and side effects were more thoroughly reported), their evolution was inconsistent. For non-small-cell lung cancer-despite the persistent lack of proof of efficacy of any active treatment-an untreated control arm was prematurely abandoned and a wide variety of tested regimens prevailed even in better-quality studies. Slightly more promising is the picture for small-cell lung cancer, where research indicates somewhat more reliable-though limited-progress. While clinical research in lung cancer has contributed little to defining the best standard care, we conclude that its heterogeneity makes it unlikely that quantitative meta-analysis of existing trials will be constructive.

Systematic evaluation of the methodology of randomized controlled trials of anticoagulation in patients with cancer

“…Our findings are consistent with those of several methodological studies in cancer [12-16,18-23]. When improvement over time has been explored, they have also found advances in some criteria, but at a rate slower than expected [16,18].…”

Evaluating non-randomised intervention studies

“…Only 23 of these included items specific to different designs; the rest included generic items to be applied to all designs. Eight of the tools were designed specifically to assess only RCTs but had been used to assess non-randomised studies, including the Chalmers scale 51 plus two modifications, 76,77 two versions of the Maastricht Criteria List, 70,71 plus three others. 46,[78][79][80] The remaining four tools were designed specifically for cohort studies only [Anders, 81 Baker, 82 Critical Appraisal Skills Programme (CASP) 64 and Newcastle-Ottawa 66 ].…”

“…These were the Chalmers scale 51 and its derivatives, 76,77 the two versions of the Maastricht Criteria List 70,71 and the JAMA Users' Guide checklist. 78,79 The Jadad 46 and McMaster 80 tools did not include items in five of the six domains.…”

“…The remaining 14 tools covered at least three core items and were considered to be the 'best' tools in our sample, two of which covered all four items. 104,105 It is interesting that of the six tools that included items in all six internal validity domains, 72,77,83,85,97,106 only one 85 included three of our four core items. None of the tools designed only for RCTs included three of the four core items.…”

“…Amongst the remaining 37 reviews, the most frequently used tools were originally developed for the assessment of RCTs (Figure 6): the Chalmers scale, 51 or modifications of it 76,77,92,96 (six reviews); the Maastricht Criteria List 69,70 (three reviews); and the Jadad scale 46 (three reviews). Other commonly used tools were ones developed for use in public health 127 (used in three reviews) and a tool developed by Oakley and colleagues 128 developed for and used by the same authors for reviews of the health promotion literature (three reviews).…”