Progress in the past decade: an examination of the precision of cluster randomized trials funded by the U.S. Institute of Education Sciences

Spybrook, Jessaca; Shi, Ran; Kelcey, Benjamin

doi:10.1080/1743727x.2016.1150454

Cited by 54 publications

(39 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Studies of this type are known as cluster-randomized trials (CRTs;Donner & Klar, 2000;Raudenbush, 1997), place-based trials (Bloom, 2005), or group-randomized trials (Murray, 1998). CRTs can provide unbiased causal inferences about the impacts of interventions in the field at larger scales, and thus generate reliable knowledge to inform evidence-based educational policies and practices (Institute of Education Sciences & National Science Foundation, 2013; Slavin, 2002;Spybrook, Shi, & Kelcey, 2016).…”

Section: Student Achievement In Elementary and Secondary Schoolmentioning

confidence: 99%

“…However, power analysis for CRTs is particularly challenging as it requires reasonable assumptions on design parameters that take into account the multilevel (i.e., nested) structure of the outcome data. The reviews on CRTs in educational research (Spybrook & Raudenbush, 2009;Spybrook, Shi, & Kelcey, 2016) indicated that most studies (between 82 and 90%) had at least three hierarchical levels (e.g., students nested within classrooms, and classrooms nested within schools), with treatment allocation at either the classroom or school level. Thus, most educational researchers conducting CRTs need multilevel design parameters that inform about the proportions of variance located at the student, classroom, and school level, as well as the respective amounts of variance that can be explained by vital covariates (e.g., pretest scores or sociodemographic characteristics) at these levels.…”

Section: Student Achievement In Elementary and Secondary Schoolmentioning

confidence: 99%

“…As listed in Table B1 Suppose that research Team 2 plans a three-level CRT to study the impact of an intervention that is intended to affect students' history achievement in comprehensive schools (grades 5 to 12). Due to budgetary constraints (see Spybrook, Shi, and Kelcey, 2016), a fixed maximum number of . = 40 schools (with -= 2 classrooms, and , = 20 students each) are at the researchers' disposal.…”

Section: Applicationsmentioning

confidence: 99%

See 2 more Smart Citations

Multilevel Design Parameters to Plan Cluster-Randomized Intervention Studies on Student Achievement in Elementary and Secondary School

Stallasch¹,

Lüdtke²,

Artelt³

et al. 2020

Preprint

View full text Add to dashboard Cite

To plan cluster-randomized trials with sufficient statistical power to detect intervention effects on student achievement, researchers need multilevel design parameters, including between-classroom and between-school differences and the amounts of variance explained by covariates at the student, classroom, and school level. Previous research has mostly been conducted in the U.S., focused on two-level designs, and limited to main achievement domains (i.e., mathematics, science, reading). Using representative data of students attending grades 1 to 12 from three German longitudinal large-scale assessments (3,963 ≤ N ≤ 14,640), we used three- and two-level latent (covariate) models to provide design parameters and corresponding standard errors for a broad array of domain-specific (e.g., mathematics, science, verbal skills) and domain-general achievement outcomes (e.g., basic cognitive functions). Three covariate sets were applied comprising (a) pretest scores, (b) sociodemographic characteristics, and (c) their combination. Design parameters varied considerably as a function of the hierarchical level, achievement outcome, and grade level. Our findings demonstrate the need to strive for an optimal fit between design parameters and target research context. The application of design parameters in power analyses is illustrated.

show abstract

Section: Student Achievement In Elementary and Secondary Schoolmentioning

confidence: 99%

Section: Student Achievement In Elementary and Secondary Schoolmentioning

confidence: 99%

Section: Applicationsmentioning

confidence: 99%

See 1 more Smart Citation

Multilevel Design Parameters to Plan Cluster-Randomized Intervention Studies on Student Achievement in Elementary and Secondary School

Stallasch¹,

Lüdtke²,

Artelt³

et al. 2020

Preprint

View full text Add to dashboard Cite

show abstract

“…The best way to assess how much impact university tutors have on interaction outcomes would be through experimental designs such as Randomised Controlled Trials (RCTs), as they are presently thought to be the most robust methodology that can 'tease' out the contribution of a specific pedagogical approach to student outcomes (Spybrook et al 2016). RCTs however could be very difficult to implement due to the complexity of the design and operational matters.…”

Section: So What Could These Metrics Be?mentioning

confidence: 99%

The Teaching Excellence Framework in the United Kingdom: An Opportunity to Include International Students as “Equals”?

Hayes

2017

Journal of Studies in International Education

View full text Add to dashboard Cite

Abstract:Research on international students in British higher education points to marginalisation of their unique perspectives in university classrooms. The aim of the paper is to consider how the most recent policy changes, particularly the Teaching Excellence Framework (TEF) continue to do so. The article also argues that the TEF, being a major higher education reform, can lead to change in attitudes towards international students and contribute to their more equal status. The paper discusses how this could be done and proposes changes to the TEF that would distance universities from policy moves that have contributed to the 'peripheral' status of international students for many years. Despite being British-based, the analysis has international implications as it points to a series of ways in which the scope and nature of metrics used in national evaluations of teaching quality can affect the status of international students in higher education.

show abstract

“…This issue also covers a range of methodological approaches, including systematic reviews and meta-analysis (Green et al 2016;Higgins and Katsipataki 2016), advancements of methods for dealing with the complexity involved in interventions by either improved models (Schweig and Pane 2016;Spybrook, Shi, and Kelcey 2016) and/or integrated approaches (Hanley, Cambers, and Haslam 2016;Peterson 2016).…”

Section: Getting These Together and Next Stepsmentioning

confidence: 99%

Is the educational ‘what works’ agenda working? Critical methodological developments

Pampaka

Williams

Homer

2016

International Journal of Research & Method in Education

View full text Add to dashboard Cite

Progress in the past decade: an examination of the precision of cluster randomized trials funded by the U.S. Institute of Education Sciences

Cited by 54 publications

References 18 publications

Multilevel Design Parameters to Plan Cluster-Randomized Intervention Studies on Student Achievement in Elementary and Secondary School

Multilevel Design Parameters to Plan Cluster-Randomized Intervention Studies on Student Achievement in Elementary and Secondary School

The Teaching Excellence Framework in the United Kingdom: An Opportunity to Include International Students as “Equals”?

Is the educational ‘what works’ agenda working? Critical methodological developments

Contact Info

Product

Resources

About