Developing a Whole Child School Screening Instrument: Evaluating Perceived Usability as an Initial Step in Planning for Consequential Validity

Abstract

We are developing the Equitable Screening to Support Youth (ESSY) Whole Child Screener to address concerns prevalent in existing school-based screenings that impede goals to advance educational equity using universal screeners. Traditional assessment development does not include end users in the early development phases, instead relying on a psychometric approach. In working to develop the ESSY Whole Child Screener, we are integrating a mixed methods approach with attention to consequential validity from the outset of measure development. This approach includes end users in measure development decisions. In this study, we interviewed a diverse sample of school staff (n = 7), administrators (n = 3), and family caregivers (n = 8) to solicit their perceptions of the usability of the initial draft of the ESSY Whole Child Screener. We identified three overarching themes: (1) paving the road for implementation of a whole child screener, (2) potential roadblocks to use, and (3) suggested paths forward to maximize positive intended consequences. Paving the road for implementation of a whole child screener includes subthemes related to alignment with existing initiatives, comprehensive yet efficient design, and potential positive consequences of assessing the whole child. Potential roadblocks to use includes subthemes of staff buy-in, family comfort with contextual screening items, teacher accuracy, and school capacity to provide indicated supports. Suggested paths forward to maximize positive intended consequences include clear and precise messaging to staff and families, optimizing instrumentation and data collection procedures, and strengthening connections to data interpretation and use. We discuss next steps in the design and testing of the initial measure as well as assessment development more broadly.