Development of the great recess framework – observational tool to measure contextual and behavioral components of elementary school recess

Abstract

BackgroundPhysical activity (PA) remains the primary behavioral outcome associated with school recess, while many other potentially relevant indicators of recess remain unexamined. Few studies have assessed observations of teacher/student interactions, peer conflict, social interactions, or safety within the recess environment. Furthermore, a psychometrically-sound instrument does not exist to examine safety, resources, student engagement, adult engagement, pro-social/anti-social behavior, and student empowerment on the playground. The purpose of the current study was to develop a valid, and reliable, assessment tool intended for use in measurement of the contextual factors associated with recess.MethodsAn iterative and multi-step process was used to develop a tool that measures safety and structure, adult engagement and supervision, student behaviors, and transitions at recess. Exploratory structural equation modeling (Mplus v. 7.4) was used to examine the underlying measurement model with observational data of the recess environment collected at 649 school-based recess periods that spanned across 22 urban/metropolitan areas in the USA. Data were also collected by two researchers at 162 recess sessions across 9 schools to examine reliability.ResultsA 17-item observation instrument, the Great Recess Framework – Observational Tool (GRF-OT), was created. Findings of exploratory structural equation modeling (ESEM) analyses supported factorial validity for a 4-factor solution and linear regressions established convergent validity where ‘structure and safety’, ‘adult engagement and supervision’, and ‘student behaviors’ were all significantly related to observed activity levels. Each sub-scale of the GRF-OT showed adequate levels of inter-rater reliability and test-retest reliability analysis indicated a higher level of stability for the GRF-OT when using a three-day average across two time points as compared to a two-day average.ConclusionsInitial evidence for a valid, and reliable, assessment tool to observationally measure the recess environment with a specific focus on safety, resources, student engagement, adult engagement, pro-social/anti-social behavior, and student empowerment was established in this study. Use of the GRF-OT can inspire evaluation, and subsequent intervention, to strategically create consistent, appropriate, and engaging school recess that impact children’s physical, cognitive, social and emotional development.Electronic supplementary materialThe online version of this article (10.1186/s12889-018-5295-y) contains supplementary material, which is available to authorized users.

Examination of the Responsiveness of the Great Recess Framework—Observational Tool

“…The range of possible scores on the GRF-OT is 17 (i.e., "1" for each item) to 68 (i.e., "4" for each item). Previous data has supported the inter-rater reliability, and the test-retest stability of the GRF-OT [5]. The GRF-OT items and scoring procedures are freely available at www.greatrecessframework.org.…”

“…Previous research [5,6] has established the reliability and validity of the GRF-OT, however the current study makes a unique contribution to the literature in examining the responsiveness of this tool to detect change over time. While levels of PA continue to be the primary outcome measure associated with school-based recess, understanding how recess may contribute to the social development of children necessitates a more holistic means of evaluation of this context.…”

“…The GRF-OT is an observational measure that is used for live data collection during recess. Previous research has shown support for the factorial validity of a four-factor model for the GRF-OT [5]. Specifically, safety and structure of the playground, adult supervision and engagement, student behaviors, and transitions have all emerged as unique constructs within the tool.…”

“…Res. Public Health 2020, 17, 225 2 of 9 (e.g., environmental, social, behavioral) is the great recess framework-observational tool (GRF-OT) developed by Massey and colleagues [5].…”

“…Response formats for all items are scores on a 1 (low quality) to 4 (high quality) rating scale, with higher scores representing higher quality markers during recess. Previous researchers [5] have established the content validity, factor structure and measurement validity, inter-rater reliability, and test-retest reliability of the GRF-OT. In doing so, it was recommended that a three-day average be used to assess recess quality, and a change score of 4.62 on the GRF-OT was needed to detect change beyond the variability expected in recess observations.…”