Using Process Data To Explain Outcomes

Abstract

The purpose of this article is to illustrate the use of process evaluation for understanding study outcomes. Data from the Child and Adolescent Trial for Cardiovascular Health (CATCH), a large school-based field trial, are used. Teacher characteristics, measures of classroom curriculum implementation, and competing influences are linked to changes in dietary knowledge, intentions, and self-efficacy of students in the intervention schools. Multiple regression analyses indicate that teacher characteristics did not predict program implementation. Teacher characteristics and program fidelity, or the number of modifications made to the classroom curriculum during implementation, had direct and independent effects on student outcomes.

The Effects of Teacher Fidelity of Implementation of Pathways to Health on Student Outcomes

“…Although we do not have specific details regarding the adaptations made, despite asking teachers to report the details of their adaptations on the self-report survey, it appears that these adaptations negatively affected program outcomes, which could be a reason we did not observe associations between implementation fidelity and program outcomes in the fourth-grade curriculum. Similarly, McGraw et al (1996) found that teachers who implemented their program made significant adaptations to their curriculum, altering the effectiveness of the study. In that case, however, the adaptations were positive and made the more program more effective (McGraw et al, 1996).…”

“…Similarly, McGraw et al (1996) found that teachers who implemented their program made significant adaptations to their curriculum, altering the effectiveness of the study. In that case, however, the adaptations were positive and made the more program more effective (McGraw et al, 1996). Based on the frequency with which adaptations are made in the adoption of prevention curriculum, program developers need to be aware of the reasons teachers make modifications and develop recommendations to ensure program goals are realized (Dusenbury et al, 2005).…”

“…Saunders, Ward, Felton, Dowda, and Pate (2006) found that a greater percentage of girls engaged in vigorous physical activity in high-implementing schools than in low-implementing schools. McGraw et al (1996) found that the percentage of classroom sessions modified by the teacher, a proxy for implementation fidelity, was associated with positive changes in student dietary self-efficacy and knowledge. Authors hypothesized that modifications were in essence tailoring the lessons to the needs of their students, making the program more effective (McGraw et al, 1996).…”

“…McGraw et al (1996) found that the percentage of classroom sessions modified by the teacher, a proxy for implementation fidelity, was associated with positive changes in student dietary self-efficacy and knowledge. Authors hypothesized that modifications were in essence tailoring the lessons to the needs of their students, making the program more effective (McGraw et al, 1996). Resnicow et al (1998) found an association between health knowledge and observed fidelity but no effects on fruit and vegetable intake or asking behaviors.…”

“…Numerous school-based health promotion and disease prevention studies have shown that program implementation is associated with program mediators (Resnicow et al, 1998; Rohrbach, Graham, & Hansen, 1993; Rohrbach, Gunning, Sun, & Sussman, 2010) and program outcomes (Durlak & DuPre, 2008; Lillehoj, Griffin, & Spoth, 2004; McGraw et al, 1996; Pentz et al, 1990). Saunders, Ward, Felton, Dowda, and Pate (2006) found that a greater percentage of girls engaged in vigorous physical activity in high-implementing schools than in low-implementing schools.…”