BACKGROUNDThe quality of randomized radiotherapy studies investigating the palliation of painful bone metastases has been questioned, with some authors recognizing the potential impact of bias on result interpretation. However, there has been no published comprehensive evaluation of quality assessment. The goals of the current study were to evaluate the quality of randomized studies using a validated checklist and to discuss implications and future directions.METHODSThe authors performed a search for studies that could be reliably assessed using the validated quality assessment instrument. Independent assessors scored study quality using the instrument.RESULTSThe median quality score of the 17 identified randomized studies was 1 of 5 (range, 0–3). The majority (71%) of points were awarded for the authors describing the study as “randomized.” The method of randomization and description of withdrawals and dropouts were scored poorly for most studies. None of the studies were awarded points for allocation concealment (blinding). The overall quality was deemed poor (a score of 0–2) for 16 of 17 (94%) studies.CONCLUSIONSThe quality of published randomized evidence comparing efficacy of fractionation schedules for the palliation of bone metastases was suboptimal. As a result of the potential biases present, subjective end points (e.g., retreatment rates) cannot be reliably evaluated. Greater efforts are required by radiation oncology trial groups to improve quality, with a particular focus on developing methods of allocation concealment and comprehensively reporting results. Cancer 2005. © 2005 American Cancer Society.