Multivariate meta-analysis of the discriminative validity of caregiver, youth, and teacher rating scales for pediatric bipolar disorder: Mother knows best about mania.

Abstract

The past 2 decades have seen a rapid increase in the amount of research on bipolar disorder in children and adolescents, including studies that look at the accuracy of symptom checklists as a way of telling if a youth might have bipolar disorder. How accurate are these checklists? Does accuracy change if they are completed by the youth or a teacher instead of the primary caregiver? Are checklists that focus specifically on symptoms of mania more accurate than checklists with more general content—typical of older measures? How much does the performance of checklists change depending on whether the sample only includes youths seeking treatment, versus including a healthy comparison group? We addressed these research questions by systematically reviewing major publication databases (PsycINFO, PubMed, and GoogleScholar) and looking at 4,094 hits based on our search. We looked for studies that reported enough information to (a) estimate the size of the difference in checklist scores (“effect size”) between cases with versus without research diagnoses of bipolar disorder for, (b) youths 18 years of age or younger, and (c) including at least 10 cases with bipolar disorder. Because we wanted to compare caregiver, teacher, and youth report on the same measures, we used a newer statistical technique, multivariate meta-analysis, to combine and compare results within as well as across studies. We found 63 effect sizes from 8 checklists used in 27 separate samples, including 11,941 youths, of whom 1,834 had diagnoses of bipolar disorder. Overall, checklists did a good job separating cases with bipolar from other youths, with an effect size of 1.05, meaning that bipolar cases scored more than a SD higher. Caregiver report was the most accurate across all checklists, performing significantly better than youth or teacher report. Scales focusing on manic symptoms also outperformed general symptom checklists. Sample composition also changed the accuracy of the checklists a great deal: Many studies either included healthy children or excluded youths with diagnoses that are difficult to tell apart from bipolar disorder. These studies gave an overly optimistic sense of how well the checklist might do at identifying youths with bipolar disorder in most clinical settings. Three checklists have shown validity in multiple studies and appear accurate enough to be helpful in improving diagnosis in clinical practice.

Diagnostic Efficiency of Caregiver Report on the SCARED for Identifying Youth Anxiety Disorders in Outpatient Settings

“…This would narrow the sources of input into the diagnosis (Campbell & Fiske, 1959), eliminating trait variance measured by youth report, interviewer behavioral observations, or clinical judgment on the part of the professional leading the consensus process. We agree with those that argue that the incorporation of these other sources of information enhances the validity of the resulting diagnosis (Bossuyt et al, 2003; Garb, 1998; Spitzer, 1983), even though the apparent effect sizes may be smaller (e.g., Youngstrom, Genzlinger, Egerton, & Van Meter, in press, found that effect sizes were +.25 d larger for parent scales when the diagnostic criterion interview only included the parent). For anxiety disorders, in particular, the use of consensus diagnoses likely increases the generalizability of results, as few clinicians would treat anxiety in the youth without first interviewing the youth and integrating their perceptions with the caregiver report (Silverman & Ollendick, 2005).…”

Diagnostic Efficiency of the Child and Adolescent Symptom Inventory (CASI-4R) Depression Subscale for Identifying Youth Mood Disorders

“…These findings are consistent with our hypothesis, as well as in line with results of previous studies (Jensen et al, 1999; Youngstrom et al, 2004; Youngstrom, Genzlinger, et al, 2015). Because the CASI-4R asks the informant to report on the child's mental state (e.g., feeling worthless/guilty, talking about death/suicide, feels inferior to others), as well as behaviors that may be more commonly noticed outside the school setting (e.g., appetite and sleep disturbance, which are omitted from the teacher form), teachers in their limited time with a student may not note changes in these areas.…”

“…AUC guidelines suggest that values in the mid .50s are considered small (not clinically useful), mid .60s are medium (may provide incremental clinical value), low to mid .70s are large (clinically informative), low .80s are excellent, and high .80s are considered exceptional under clinically rigorous designs (highly informative) (Rice & Harris, 2005). Values higher than .90 in mental health research often indicate a design flaw, such as using a comparison group that is too distinct or that excludes common conditions that might create false positives (e.g., healthy controls; Youngstrom, Genzlinger, Egerton, & Van Meter, 2015; Youngstrom, Meyers, Youngstrom, Calabrese, & Findling, 2006). …”

Development of an Objective Autism Risk Index Using Remote Eye Tracking

“…Comparisons that use healthy controls produce much larger effect sizes that are prone to greater shrinkage when the same test is used in clinical settings where other developmental conditions that could have high false positive rates are common. 49 …”