Evaluation of a statewide dissemination and implementation of physical activity intervention in afterschool programs: a nonrandomized trial

Abstract

In 2015, YMCA-operated afterschool programs (ASPs) across South Carolina pledged to achieve the national standard that calls for every child to accumulate 30 min/day of moderate-to-vigorous physical activity (MVPA) during program time. This study shares the first-year findings related to the dissemination, implementation, and outcomes associated with the statewide intervention to achieve the MVPA Standard. Twenty ASPs were sampled from all YMCA-operated ASPs (N = 97) and visited at baseline (spring 2015) and first-year follow-up (spring 2016). Programs were provided standardized professional development training to increase the MVPA children accumulated while attending ASPs. The training focused on extending the scheduled time for activity opportunities and modifying commonly played games to increase MVPA. The RE-AIM framework was used to evaluate the statewide intervention. Accelerometer-derived MVPA was the primary outcome. Implementation was evaluated via direct observation. Intent-to-treat (ITT) and as-treated comparisons were conducted in summer 2016. Reach/adoption was variable, with attendance at trainings ranging from 0 to 100% across ASPs. Effectiveness of the intervention using ITT models indicated no changes from baseline in the percentage of programs meeting the MVPA standard for boys or girls. Implementation levels also varied and were related to increases in both boys' and girls' MVPA for moderate and high implementers. Findings indicate improvements in MVPA can be made from attending the trainings and implementing some or all of the training components. Additional work is necessary to identify ways to ensure staff attend trainings to implement strategies and to identify which specific factors contributed to increases in MVPA.

Identification and evaluation of risk of generalizability biases in pilot versus efficacy/effectiveness trials: a systematic review and meta-analysis

“…Our findings highlight that intervention researchers need to carefully consider whether information obtained from pilot tests of an intervention delivered by highly trained research team members, with extensive support for intervention delivery, over short timeframes with different measures than are to be used in the larger-trial can be sustained and is consistent with what is intended to-be-delivered in the efficacy/effectiveness trial. Including one or more of these biases in a pilot study could result in inflated estimates of effectiveness during the pilot and lead interventionists to believe the intervention is more effective than the actual effect achieved when delivered in a efficacy/effectiveness trial without these biases [14,26,166]. These are critical decisions because, if the purpose of a pilot study is to determine whether a large-scale trial is warranted, yet the outcomes observed from the pilot study are contingent upon the features included in the pilot that are not intended to be or cannot be carried forward in an efficacy/effectiveness trial, the likelihood of observing limited or null results in the efficacy/effectiveness trial is high.…”

Perspectives on program mis-implementation among U.S. local public health departments

“…Reassignment of staff, changes in staff workloads, and changes in leadership led to discontinuation of an evidence-based trauma intervention in a large urban school district [48]. Lack of organizational policy to support staff time to attend training led to partial implementation of after school physical activity efforts [49]. But in the present study, ending of funding was by far the most commonly reported reason for inappropriate termination, as found in a recent review [50], and organizational supports were not protective.…”

“…Public health systems are working to scale up EBIs [22,45,49], but little is known about strategies to address inappropriate continuation of ineffective approaches. Here public health can learn from medical studies, even though organizational structures and funding sources differ.…”

Physical Activity and Sedentary Behavior of Children in Afterschool Programs: An Accelerometer-Based Analysis in Full-Day and Half-Day Elementary Schools in Germany

“…Several studies investigated PA and SB in the school context, especially considering full-day school's ASPs: Beets et al ( 17 ) showed in their study with more than 1,000 elementary school children attending 97 ASPs in South Carolina, USA, operated by the Young Men's Christian Association (YMCA), that only one quarter reached the 30-min guideline of PA in ASPs. On average, children spent 21.4 min in MVPA and 64.3 min in SB at baseline.…”