Getting Off the “Gold Standard”: Randomized Controlled Trials and Education Research

Sullivan, Gail M.

doi:10.4300/jgme-d-11-00147.1

Cited by 190 publications

(145 citation statements)

References 17 publications

Supporting

Mentioning

143

Contrasting

Unclassified

Order By: Relevance

“…In educational research it is difficult to tightly control variables and blind subjects in ways that are ethical or feasible, and one cannot 'apply curriculum daily' in the same way that one can prescribe medication. [5] It is therefore impossible to control for confounding variables in learning environments, making the attempt to use RCTs in this context 'hopelessly flawed' . [10] The beliefs of the researcher around the context in which the research will take place clearly inform the choice of method used to gather data, which in turn informs the outcome of the study.…”

Section: Educational and Clinical Contexts Differmentioning

confidence: 99%

“…By controlling for the influence of all other variables between groups -through the assumed equal distribution of those differences in a large enough randomised sample -any differences in outcome can be more confidently attributed to the intervention. [5] As the RCT is a powerful tool successfully used in medical and clinical research, some have suggested that it should therefore underpin all 'good' research regardless of context' . [9] However, one cannot assume that RCTs can provide more -and better -evidence, which inevitably leads to improvements in education.…”

Section: Rcts In Educational Researchmentioning

confidence: 99%

“…[6] The factors that influence learning outcomes cannot be recreated in different contexts, or even in the same context at different times. [5] Education therefore exists in an open system, and even if we attempt to reduce and limit change and variation -internally and externally -we will not be able to determine causality. [15] RCTs assume that every implementation of the intervention is the same, that everyone receiving the intervention will be affected in the same way, and that giving and receiving the intervention is divorced from the reality of the individual personalities and institutional contexts of the participants.…”

Section: Educational and Clinical Contexts Differmentioning

confidence: 99%

“…These include for example, factors that lead to differences in studying methods, changes in learner motivation, and effects of other, non-intervention experiences that occur during the implementation of the intervention. [5] It is impossible to create a closed education system, even an 'artificially closed system' , [15] and therefore almost impossible to identify how much of the intervention the learner actually 'receives' , or to determine what the learner does with what the teacher provides. [6] The factors that influence learning outcomes cannot be recreated in different contexts, or even in the same context at different times.…”

Section: Educational and Clinical Contexts Differmentioning

confidence: 99%

“…[4] There is no doubt that the method has utility in determining cause-effect relationships between medical treatments and patient outcomes, making it a powerful design for intervention studies with the objective of determining the influence of one variable on another. [5] In an educational context it may initially seem reasonable to expect that an experimental design could determine the effect of a teaching intervention that aims to improve student learning. The argument is that by using randomisation to average out the differences between students, one would be able to demonstrate which teaching and learning strategies lead to the largest effect sizes.…”

mentioning

confidence: 99%

See 4 more Smart Citations

Randomised controlled trials in educational research: Ontological and epistemological limitations

Rowe

Oltmann

2016

AJHPE

View full text Add to dashboard Cite

Evidence-based practice (EBP) is positioned as an inherent good in the medical and clinical literature, and not without reason. It relies on the integration of research evidence, clinical expertise and patient preferences, [1] and has become a foundation on which health systems are built and improved. There is however, a growing body of literature that takes a more critical stance towards EBP, especially when practitioners make assumptions about what constitutes 'the evidence' and how the data informing that evidence are gathered. [2,3] The evidence upon which EBP is premised is usually derived from experimental research conducted in professional disciplines that are firmly rooted in the positivist paradigm; the research method most closely associated with this is the randomised controlled trial (RCT). RCTs are quantitative, controlled experiments in which the effect of an intervention can be determined more objectively than by observational studies. [4] There is no doubt that the method has utility in determining cause-effect relationships between medical treatments and patient outcomes, making it a powerful design for intervention studies with the objective of determining the influence of one variable on another. [5] In an educational context it may initially seem reasonable to expect that an experimental design could determine the effect of a teaching intervention that aims to improve student learning. The argument is that by using randomisation to average out the differences between students, one would be able to demonstrate which teaching and learning strategies lead to the largest effect sizes. These data, presumed to be free of subjective interpretation, could then inform policies that drive the implementation of effective teaching interventions. [6] However, if we assume that the evidence gathered via experimental research provides insight into an objective reality, we must take a position on teaching and learning that is at odds with our best explanations for how learning happens. Therefore, if we want to use RCTs in educational research, we must assume that there is a cause-effect relationship in the teaching and learning interaction that can be objectively measured. In this article we argue that RCTs are an inappropriate design choice for educational research because they force one to assume ontological and epistemological positions that are at odds with theoretically informed perspectives of learning. RCTs in educational researchWe begin by highlighting the biased way in which RCTs are positioned relative to other forms of research endeavours, explicit in the language employed by RCT proponents. Goldacre [7] suggests that 'Evidence-based interventions in teaching could … replace the current system where untested methods are passed to teachers through a variety of often dubious outlets' , and 'We need a slow revolution that puts evidence at the heart of teaching' . Torgerson [8] asserts that RCTs are the ultimate expression of evaluative research, referring to 'the importance and supremacy of the RCT'...

show abstract

Section: Educational and Clinical Contexts Differmentioning

confidence: 99%

Section: Rcts In Educational Researchmentioning

confidence: 99%

Section: Educational and Clinical Contexts Differmentioning

confidence: 99%

Section: Educational and Clinical Contexts Differmentioning

confidence: 99%

mentioning

confidence: 99%

See 3 more Smart Citations

Randomised controlled trials in educational research: Ontological and epistemological limitations

Rowe

Oltmann

2016

AJHPE

View full text Add to dashboard Cite

show abstract

How comprehensive are research studies investigating the efficacy of technology‐enhanced learning resources in anatomy education? A systematic review

Clunie

Morris

Joynes

et al. 2017

Anatomical Sciences Ed

View full text Add to dashboard Cite

Anatomy education is at the forefront of integrating innovative technologies into its curricula. However, despite this rise in technology numerous authors have commented on the shortfall in efficacy studies to assess the impact such technology-enhanced learning (TEL) resources have on learning. To assess the range of evaluation approaches to TEL across anatomy education, a systematic review was conducted using MEDLINE, the Educational Resources Information Centre (ERIC), Scopus, and Google Scholar, with a total of 3,345 articles retrieved. Following the PRISMA method for reporting items, 153 articles were identified and reviewed against a published framework-the technology-enhanced learning evaluation model (TELEM). The model allowed published reports to be categorized according to evaluations at the level of (1) learner satisfaction, (2) learning gain, (3) learner impact, and (4) institutional impact. The results of this systematic review reveal that most evaluation studies into TEL within anatomy curricula were based on learner satisfaction, followed by module or course learning outcomes. Randomized controlled studies assessing learning gain with a specific TEL resource were in a minority, with no studies reporting a comprehensive assessment on the overall impact of introducing a specific TEL resource (e.g., return on investment). This systematic review has provided clear evidence that anatomy education is engaged in evaluating the impact of TEL resources on student education, although it remains at a level that fails to provide comprehensive causative evidence. Anat Sci Educ 11: 303-319. © 2017 American Association of Anatomists.

show abstract

Effectiveness of team teaching in biochemistry lectures for undergraduate students

Govindarajan

Rajaragupathy

Subramanian³

et al. 2021

Biochem Molecular Bio Educ

View full text Add to dashboard Cite

Team teaching is an innovative method to engage the large group and make lectures more interesting. The aim of the study is to evaluate the impact of team teaching in first year biochemistry lecture classes. After obtaining ethical clearance, 150 first year medical undergraduate students participated in the study on a voluntary basis. Topic identified for team teaching was "Protein biosynthesis" in Biochemistry, which was designed to be carried out in 5 h of lecture by three teachers. At the end of the team teaching module and traditional lectures, a structured anonymous feedback was obtained from the students. At the end of each class, students' understanding of the lecture was assessed by an MCQ test comprising 5-6 items, which mainly tested the recollection and understanding levels of Bloom's taxonomy. This was compared with traditional lectures. The mean MCQ test score was significantly higher for team teaching when compared to traditional teaching. The students had increased attention span and were able to answer questions from later part of the class in team lectures. The students had reported that lectures taught by team were more interesting and informative. They also felt that the team was more receptive to the

show abstract

Getting Off the “Gold Standard”: Randomized Controlled Trials and Education Research

Cited by 190 publications

References 17 publications

Randomised controlled trials in educational research: Ontological and epistemological limitations

Randomised controlled trials in educational research: Ontological and epistemological limitations

How comprehensive are research studies investigating the efficacy of technology‐enhanced learning resources in anatomy education? A systematic review

Effectiveness of team teaching in biochemistry lectures for undergraduate students

Contact Info

Product

Resources

About