No abstract

The interpretation of between-group comparisons is facilitated by the creation of treatment groups that are similar to each other in baseline composition. To prevent treatment effects from being confounded with time effects, most trials use restricted randomization to force balance. An unintended consequence of these restrictions is that they create patterns that allow for the prediction of future treatment allocations, and hence selection bias, especially in unmasked trials. In fact, the more restrictive the allocation procedure, the greater the potential for selection bias. It was decided, in the context of a recent clinical trial comparing two dosing schedules of paclitaxel and carboplatin for advanced stage IIIB/IV non-small-cell lung cancer, that the randomized block procedure could not simultaneously protect sufficiently against both selection and chronological bias. In this paper we detail our development of the maximal procedure. The maximal procedure takes as input the extent of chronological bias allowed by the randomized block procedure, then matches it, but does so with fewer restrictions. This feature makes the maximal procedure more resistant to selection bias than the randomized block procedure is.

Flawed evaluation of clinical trial quality allows flawed trials to thrive (get funded, obtain IRB approval, get published, serve as the basis of regulatory approval, and set policy). A reasonable evaluation of clinical trial quality must recognize that any one of a large number of potential biases could by itself completely invalidate the trial results. In addition, clever new ways to distort trial results toward a favored outcome may be devised at any time. Finally, the vested financial and other interests of those conducting the experiments and publishing the reports must cast suspicion on any inadequately reported aspect of clinical trial quality. Putting these ideas together, we see that an adequate evaluation of clinical quality would need to enumerate all known biases, update this list periodically, score the trial with regard to each potential bias on a scale of 0% to 100%, offer partial credit for only that which can be substantiated, and then multiply (not add) the component scores to obtain an overall score between 0% and 100%. We will demonstrate that current evaluations fall well short of these ideals.