Objectives To evaluate the risk of bias tool, introduced by the Cochrane Collaboration for assessing the internal validity of randomised trials, for inter-rater agreement, concurrent validity compared with the Jadad scale and Schulz approach to allocation concealment, and the relation between risk of bias and effect estimates. Design Cross sectional study. Study sample 163 trials in children. Main outcome measures Inter-rater agreement between reviewers assessing trials using the risk of bias tool (weighted κ), time to apply the risk of bias tool compared with other approaches to quality assessment (paired t test), degree of correlation for overall risk compared with overall quality scores (Kendall's τ statistic), and magnitude of effect estimates for studies classified as being at high, unclear, or low risk of bias (metaregression). Results Inter-rater agreement on individual domains of the risk of bias tool ranged from slight (κ=0.13) to substantial (κ=0.74). The mean time to complete the risk of bias tool was significantly longer than for the Jadad scale and Schulz approach, individually or combined (8. 8 minutes (SD 2.2) per study v 2.0 (SD 0.8), P<0.001). There was low correlation between risk of bias overall compared with the Jadad scores (P=0.395) and Schulz approach (P=0.064). Effect sizes differed between studies assessed as being at high or unclear risk of bias (0.52) compared with those at low risk (0.23). Conclusions Inter-rater agreement varied across domains of the risk of bias tool. Generally, agreement was poorer for those items that required more judgment. There was low correlation between assessments of overall risk of bias and two common approaches to quality assessment: the Jadad scale and Schulz approach to allocation concealment. Overall risk of bias as assessed by the risk of bias tool differentiated effect estimates, with more conservative estimates for studies at low risk.

BackgroundMuch controversy exists regarding the clinical efficacy of behavioural and developmental interventions for improving the core symptoms of autism spectrum disorders (ASD). We conducted a systematic review to summarize the evidence on the effectiveness of behavioural and developmental interventions for ASD.Methods and FindingsComprehensive searches were conducted in 22 electronic databases through May 2007. Further information was obtained through hand searching journals, searching reference lists, databases of theses and dissertations, and contacting experts in the field. Experimental and observational analytic studies were included if they were written in English and reported the efficacy of any behavioural or developmental intervention for individuals with ASD. Two independent reviewers made the final study selection, extracted data, and reached consensus on study quality. Results were summarized descriptively and, where possible, meta-analyses of the study results were conducted. One-hundred-and-one studies at predominantly high risk of bias that reported inconsistent results across various interventions were included in the review. Meta-analyses of three controlled clinical trials showed that Lovaas treatment was superior to special education on measures of adaptive behaviour, communication and interaction, comprehensive language, daily living skills, expressive language, overall intellectual functioning and socialization. High-intensity Lovaas was superior to low-intensity Lovaas on measures of intellectual functioning in two retrospective cohort studies. Pooling the results of two randomized controlled trials favoured developmental approaches based on initiative interaction compared to contingency interaction in the amount of time spent in stereotyped behaviours and distal social behaviour, but the effect sizes were not clinically significant. No statistically significant differences were found for: Lovaas versus special education for non-verbal intellectual functioning; Lovaas versus Developmental Individual-difference relationship-based intervention for communication skills; computer assisted instruction versus no treatment for facial expression recognition; and TEACCH versus standard care for imitation skills and eye-hand integration.ConclusionsWhile this review suggests that Lovaas may improve some core symptoms of ASD compared to special education, these findings are based on pooling of a few, methodologically weak studies with few participants and relatively short-term follow-up. As no definitive behavioural or developmental intervention improves all symptoms for all individuals with ASD, it is recommended that clinical management be guided by individual needs and availability of resources.

Evidence from available trials shows no effect of vitamin D3 supplementation on glucose homeostasis or diabetes prevention. Definitive conclusions may be limited in the context of the moderate degree of heterogeneity, variable risk of bias, and short-term follow-up duration of the available evidence to date.