Validating Screening Questionnaires for Internalizing and Externalizing Disorders against Clinical Interviews in 8 to 17-Year-Old Syrian Refugee Children

Abstract

Syrian children affected by the civil war are at increased risk of mental health problems, including depression, anxiety, post-traumatic stress disorder (PTSD), and externalizing behaviour problems. Screening questionnaires are designed to identify individual children who require further assessment and treatment, and also estimate the need for mental health services in a population. However, few questionnaires have been rigorously tested in this population. This study examined the reliability and validity of questionnaires for depression (Center for Epidemiological Studies Depression Scale for Children, CES-DC, self-report, 10-item version), anxiety (Screen for Child Anxiety Related Emotional Disorders, SCARED, self-report, 18-item version), PTSD (Child PTSD Symptom Scale, CPSS, self-report), and internalizing and externalizing behavior problems (Strengths and Difficulties Questionnaire, SDQ, parent-report version) in a population sample of 8-17 year old Syrian children living in Informal Tented Settlements (ITS) in the Beqaa region of Lebanon. In addition, several ways of measuring functional impairment due to mental health problems were compared. These included self- and parent-report questionnaires (World Health Organization Disability Assessment Schedule, WHODAS-Child; SDQ Impact supplement, parent-report only) and an interviewer rating of severity (Clinical Global Impression–severity, CGI-s). Questionnaires were translated into Arabic and modified based on pilot testing with Syrian children. Responses from N=1006 children and caregivers were used for analysis, a subset of whom had additional clinical interview data (MINI KID + clinical judgement; N=119). The self-report questionnaires showed good internal consistency reliability with alpha>.80, though the parent-report SDQ and WHODAS-Child fell below this level. In terms of validity, the SDQ externalizing scale performed well in differentiating children with conduct problems from those without and it was possible to achieve a fair balance between sensitivity (82%) and specificity (71%). The CES-DC, CPSS, SDQ total difficulties, and WHODAS-Child (selfreport) achieved an acceptable level of validity, though it was harder to achieve a good balance between sensitivity and specificity. In most cases, at least 50% of those screening positive were false positives, meaning that a more in-depth follow up assessment would be required if these tools were used as screeners in a clinical setting. Furthermore, correctionwould be needed if used to estimate prevalence rates for mental disorders in this population. There was moderate convergent validity between measures of functional impairment, with self-report WHODAS-Child showing greater agreement with interviewer ratings when compared to parent-report measures (WHODAS and SDQ Impact). Measuring functional impairment and distress due to mental health problems should help to differentiate children with clinically significant mental health problems from those with subthreshold problems; however, more work will be required to establish how helpful the tools used here are in achieving that aim.

The dynamic nature of refugee children's resilience: a cohort study of Syrian refugees in Lebanon

“…However, specificity fell below 80%, and consequently the high-risk groups may contain some false positives. Prevalence estimates adjusted for false positives and negatives are therefore lower than reported here (McEwen et al, 2022a), but adjustments cannot be applied at the individual level, so we retain unadjusted estimates. Secondly, PTSD, depression, and externalising problems may be differentially associated with some of the factors measured.…”

“…Following pilot testing with Syrian refugees in Lebanon, the CES-DC was abridged to ten items and minor changes to phrasing (including Arabic dialect) were made to the CES-DC and CPSS (McEwen et al, 2020; online Supplementary 1.1). Cut-off scores on each outcome (12 out of 51 on the adjusted CPSS, 10 out of 30 on the adjusted CES-DC, and 12 out of 44 on the combined externalising scale total) were derived from structured clinical interviews (MINI-KID, Sheehan et al, 2010) and clinical judgement in a representative subsample (n = 119) of the cohort (McEwen et al, 2020). Cut-offs had sensitivity of 81-85%, but specificity fell below 80%, meaning that some children flagged as at risk may not represent clinical cases.…”

“…However, scales were extensively piloted and, where possible, modified to be context-appropriate. Furthermore, we derived cut-offs through clinical assessment in a subsample, choosing cut-offs with the best balance of sensitivity and specificity for our particular sample (McEwen et al, 2020). However, specificity fell below 80%, and consequently the high-risk groups may contain some false positives.…”

War exposure, post-traumatic stress symptoms and hair cortisol concentrations in Syrian refugee children

“…Child-reported PTSD symptoms were assessed using the Child PTSD Symptom Scale (CPSS), with minor changes made to phrasing and Arabic dialect following piloting. This scale had good validity in similar populations and showed acceptable reliability and validity against DSM-5 diagnosis of PTSD assessed via clinical interview in a representative subsample of the BIOPATH cohort [42].…”

Predictors of psychological risk and resilience among Syrian refugee children

“…Outcomes of interest were self‐reported PTSD (Child PTSD Symptom Scale; CPSS; Foa, Johnson, Feeny, & Treadwell, 2001), self‐reported depression (Centre for Epidemiological Studies Depression Scale for Children; CES‐DC; Faulstich, Carey, Ruggiero, Enyart, & Gresham, 1986), and caregiver‐reported externalising behaviour problems measured using the externalising subscale of the Strengths and Difficulties Questionnaire (SDQ; Goodman, 1997) and additional items related to conduct disorder and oppositional defiant disorder administered separately (McEwen et al., 2022). Scales were chosen according to reliability and validity in similar populations and pilot tested with Syrian refugees in Lebanon (McEwen et al., 2020; Appendix S1, Supplemental Methods). The CES‐DC was abridged to 10 items (Table S2) and minor changes to phrasing (including Arabic dialect) were made to the CES‐DC and CPSS based on pilot data (Appendix S1, Supplemental Methods; Table S1).…”