Program Strength and Fidelity in Evaluation

Summerfelt, Wm. Thomas

doi:10.1207/s1532480xads0702_2

Cited by 37 publications

(46 citation statements)

References 61 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…As a result, it often falls to the evaluator to grapple with questions about how fidelity should be understood and assessed in the context of a given intervention (Goodson, Price, & Darrow, 2015;Century, Rudnick, & Freeman, 2010;Summerfelt, 2003 (Clay, 2005). The Standards and Guidelines are structured according to the key implementer roles in Reading Recovery: teacher leader, teacher, site coordinator, and university trainer.…”

Section: Defining Implementation Fidelity Defining Implementation Fidmentioning

confidence: 99%

Reading Recovery: An Evaluation of the Four-Year i3 Scale-Up

May¹,

Sirinides²,

Gray³

et al. 2016

View full text Add to dashboard Cite

CPRE released its evaluation of one of the most ambitious and well-documented expansions of a U.S. instructional curriculum. The rigorous independent evaluation of the Investing in Innovation (i3) scale-up of Reading Recovery, a literacy intervention for struggling first graders, was a collaboration between CPRE and the Center for Research on Education and Social Policy (CRESP) at the University of Delaware.The CPRE/CRESP evaluation revealed that students who participated in Reading Recovery significantly outperformed students in the control group on measures of overall reading, reading comprehension, and decoding. These effects were similarly large for English language learners and students attending rural schools, which were the student subgroups of priority interest for the i3 scale-up grant program.The study included an in-depth analysis of program implementation. Key findings focus on the contextual factors of the school and teachers that support the program's success and the components of instructional strength in Reading Recovery. Disciplines Curriculum and Instruction | Educational Assessment, Evaluation, and Research | Educational Methods | Education Policy | Elementary and Middle and Secondary Education Administration | Elementary Education and Teaching | Reading and Language CommentsView on the CPRE website.This report is available at ScholarlyCommons: http://repository.upenn.edu/cpre_researchreports/81 CONSORTIUMforPOLICYR ESEARCHinEDUCATION A R E S E A R C H R E P O R T About the Center for Research in Education & Social Policy (CRESP)The Center for Research in Education and Social Policy (CRESP) within the College of Education and Human Development at the University of Delaware conducts rigorous research to help policymakers and practitioners in education, health care and human services determine which policies and programs are most promising for improving outcomes in children, youth, adults and families.Although research in prevention sciences and health care have long used rigorous designs to assess the effectiveness of programs, it was not until the Education Sciences Reform Act of 2002 that we witnessed a dramatic increase in the quantity and quality of research to evaluate the effects of education programs and policies. The education community began to focus on research that could measure the impact of these programs through randomized experiments and other research designs that support causal conclusions and can determine whether, how well, for whom, and why new programs and interventions work.CRESP specializes in experimental and quasi-experimental research that uses quantitative and mixed methods to evaluate how and how well programs and interventions work to improve educational, family, and health outcomes in schools and communities.

show abstract

Section: Defining Implementation Fidelity Defining Implementation Fidmentioning

confidence: 99%

Reading Recovery: An Evaluation of the Four-Year i3 Scale-Up

May¹,

Sirinides²,

Gray³

et al. 2016

View full text Add to dashboard Cite

show abstract

“…In order to do this, we have worked to develop a comprehensive understanding, and an approach to measuring, Reading Recovery's implementation. In the past, discussions of implementation in the context of experimental studies have generally been framed in terms of fidelity (Century et al, 2010;Bauman, Stein, & Ireys, 1991;Summerfelt, 2003;Hulleman & Cordray, 2009;Weiss, Bloom, & Brock, 2013); the implicit presumption of the bulk of this work is that fidelity is an adequate measure of implementation. However, we have consistently observed that implementation of Reading Recovery is more complex than can be understood by an exclusive focus on fidelity.…”

Section: Ongoing Questionsmentioning

confidence: 99%

“…In parsing implementation fidelity, it seems, it falls to the evaluator to grapple with questions about how fidelity should be defined in the context of a given intervention and where fidelity to the program model begins and ends (Century, Rudnick, & Freeman, 2010;Summerfelt, 2003).…”

mentioning

confidence: 99%

Evaluation of the i3 Scale-up of Reading Recovery | Year Two Report, 2012-13

May¹,

Goldsworthy²,

Armijo³

et al. 2014

View full text Add to dashboard Cite

Reading Recovery is a short-term early intervention designed to help the lowest-achieving readers in first grade reach average levels of classroom performance in literacy. Students identified to receive Reading Recovery meet individually with a specially trained Reading Recovery teacher every school day for 30-minute lessons over a period of 12 to 20 weeks. The purpose of these lessons is to support rapid acceleration of each child's literacy learning. In 2010, The Ohio State University received a Scaling Up What Works grant from the U.S. Department of Education Investing in Innovation (i3) Fund to expand the use of Reading Recovery across the country. The award was intended to fund the training of 3,675 new Reading Recovery teachers in U.S. schools, thereby expanding service to an additional 88,200 students.The Consortium for Policy Research in Education (CPRE) was contracted to conduct an independent evaluation of the i3 scale-up of Reading Recovery over the course of five years. The evaluation includes parallel rigorous experimental and quasi-experimental designs for estimating program impacts, coupled with a largescale mixed-methods study of program implementation. This report presents the findings of the second year of the evaluation. The primary goals of this evaluation are: a) to provide experimental evidence of the impacts of Reading Recovery on student learning under this scale-up effort ; b) to assess the success of the scale-up in meeting the i3 grant's expansion goals; and c) to document the implementation of the scale-up and fidelity to program standards.This document is the second in a series of three reports based on our external evaluation of the Reading Recovery i3 Scale-Up. This report presents results from the impact and implementation studies conducted over the 2012-2013 school year-the third year of the scale-up effort and the second full year of the evaluation.In order to estimate the impacts of the program, a sample of first graders who had been selected to receive Reading Recovery were randomly assigned to a treatment group that received Reading Recovery immediately, or to a control group that did not receive Reading Recovery until the treatment group had exited the intervention. The reading achievement of students in this sample was assessed using a standardized assessment of reading achievement-the Iowa Tests of Basic Skills (ITBS). The data for the implementation study include extensive interviews and surveys with Reading Recovery teachers, teacher leaders, site coordinators, University Training Center directors, members of the i3 project leadership team at The Ohio State University, and principals and first-grade teachers in schools involved in the scale-up. Case studies were also conducted in nine i3 scale-up schools to observe how Reading Recovery operates in different contexts. Disciplines About the Center for Research in Education & Social Policy (CRESP)The Center for Research in Education and Social Policy (CRESP) within the College of Education and Human Development at ...

show abstract

“…Dane and Schneider 1998, Summerfelt 2003, Dusenbury et al 2003. Dane and Schneider (1998) identified adherence, exposure, quality of delivery, participant responsiveness and program differentiation as five domains of fidelity.…”

mentioning

confidence: 99%

Towards an understanding of fidelity within the context of school-based health education

McCuaig

Hay

2013

Critical Public Health

View full text Add to dashboard Cite

Schools and schooling have long provided a tempting site for the delivery of public health strategies that address and promote young people's current and future health. However, an emerging concern regarding the mobilisation of public health interventions within school settings has been the failure of school teachers to deliver such programs with fidelity. For educators, these notions of fidelity stand in stark contrast to the tenets of student-centred teaching. In seeking to explore these tensions further, this paper draws upon a collaborative health education project conducted with schools and teachers from Queensland, Australia.Findings from this project reveal the complexity associated with curriculum implementation in school settings, where diverse resources including timetable allocations and teacher expertise mitigate the achievement of program fidelity. In our efforts to explain the findings emerging from this project, we have drawn on the conceptual reference points of Basil Bernstein's theory of the pedagogic device to reveal the predictable misalignment of the health and education sectors' expected outcomes of school based health initiatives. In conclusion, we argue that our exploration of issues pertaining to fidelity demonstrates the need for health and education sectors alike to conduct their work according to a clear articulation of the realistic, educative role that schools can play in promoting healthy living.

show abstract

Program Strength and Fidelity in Evaluation

Cited by 37 publications

References 61 publications

Reading Recovery: An Evaluation of the Four-Year i3 Scale-Up

Reading Recovery: An Evaluation of the Four-Year i3 Scale-Up

Evaluation of the i3 Scale-up of Reading Recovery | Year Two Report, 2012-13

Towards an understanding of fidelity within the context of school-based health education

Contact Info

Product

Resources

About