A General Framework for the Evaluation of Clinical Trial Quality

Abstract

Flawed evaluation of clinical trial quality allows flawed trials to thrive (get funded, obtain IRB approval, get published, serve as the basis of regulatory approval, and set policy). A reasonable evaluation of clinical trial quality must recognize that any one of a large number of potential biases could by itself completely invalidate the trial results. In addition, clever new ways to distort trial results toward a favored outcome may be devised at any time. Finally, the vested financial and other interests of those conducting the experiments and publishing the reports must cast suspicion on any inadequately reported aspect of clinical trial quality. Putting these ideas together, we see that an adequate evaluation of clinical quality would need to enumerate all known biases, update this list periodically, score the trial with regard to each potential bias on a scale of 0% to 100%, offer partial credit for only that which can be substantiated, and then multiply (not add) the component scores to obtain an overall score between 0% and 100%. We will demonstrate that current evaluations fall well short of these ideals.

Critical Appraisal of Effectiveness of Oral Fingolimod on Relapsing Multiple Sclerosis

“…It means that we have to treat 5 patients with multiple sclerosis to prevent one relapse during the treatment period. The NNT between 2-5 indicates that the medication is effective (Berger and Alperson, 2009). Fingolimod oral administration is simpler than subcutaneous interferon injection.…”

“…We limited our searches only for articles that were full text, published within the last 10 years, and in English. We used the Jadad scale to measure the quality of the evidences (Berger and Alperson, 2009). For the clinical efficacy section, only clinical trials published in English and indexed on MEDLINE, evaluating efficacy of fingolimod with reduction of symptoms or improvement in disease state as an endpoint were included.…”

La eficacia de los protocolos de intervención con enfoque familiar para el tratamiento de los adolescentes consumidores de drogas: una revisión sistemática

“…Esse instrumento foi escolhido porque é de fácil e rápida aplicação e é um dos recursos mais utilizados para avaliar a qualidade de ensaios clínicos randomizados (Berger & Alperson, 2009). A escala JADAD é dividida em duas partes.…”

Letter to the Editor: Efficacy and Degree of Bias in Knee Injury Prevention Studies: A Systematic Review of RCTs

“…This would be a good start, necessary but not sufficient, as even CONSORT-compliant trials can be fatally flawed. No check list can anticipate every possible way a trial may be biased, but some are better than others, and the check list used as a first pass to check for trial quality and integrity must evolve to keep up with changes in trial design, as these changes introduce opportunities for new biases to arise [3]. Researchers and statisticians must remain vigilant to new biases, and always keep an eye open to the need to augment CONSORT or any other check list; the soccer players and other athletes must recognize flawed research for what it is, take it for what it is worth, and, until more definitive evidence can be produced, keep on kicking.…”