Is experimental research passé

Abstract

In perusing the contents of this issue, I discovered that, once again, the majority of the papers are not experimental. There are several qualitative studies-of teachers' attitudes to student feedback, reflective learning for continuing education, residents' perceptions of outpatient teaching, and assessment expert's view of the framework for assessments, cohort study looking at predictors of motivation for medicine, a survey of teachers' perceptions of student feedback, a correlational study of concurrent validity of a personality test, a review paper on the issue of whether Asian students are rote learners, and 3 randomized trials. One looked at the use of simulation in addition to a PBL session for management of respiratory and cardiac distress, another at web-based learning, and a third examined the effect of testing on retention of CPR skills. So of the 11 original studies, 3 were experimental.I suppose I should be pleased. Todres et al. (2007), in a widely cited review of 2 years of Medical Education and Medical Teacher articles, found only 5% were randomized controlled trials. So at 3/11 = 27%, we're doing much better.Better? Pardon a second. Should we be striving for 100% randomized trials? Is this any measure of quality of research? After all, a study of faculty perception of student feedback is unlikely to benefit from having a control group (randomized of course) who were not given feedback and then asked their perceptions of not getting feedback. For that matter, randomizing students to be Asian or not is a bit impractical. In short, the variety of research designs in the papers in this issue reflect the nature of the research questions asked, which really span the gamut of educational research, and only a few are amenable to experimentation.But what of quality? One carryover from clinical research is a perverse hierarchy of research designs where randomized controlled trial is at the top in quality, case series is at the bottom, and cohort studies are somewhere in the middle. The first problem with this approach is that it is a very limited taxonomy, suitable for epidemiologic studies and little else. The common approach to a study of assessment, looking at reliability and validity,

Locating “gold standard” evidence for simulation as a substitute for clinical practice in prelicensure health professional education: A systematic review

“…The “gold standard” in scientific research is generally based on the RCT. While there is some debate about the application of this standard in educational research (Norman, ; Sullivan, ), we support the assertion that RCTs have a role to play in education research when examining relatively standardised interventions such as clinical simulation, when they reflect the nature of the research questions asked and are amenable to experimentation (Norman, ). However, to date the use of RCTs in simulation education research is relatively sparse; thus, we opted to include other levels of evidence in this systematic review to more fully answer the research questions.…”

Didactic CME and practice change: don’t throw that baby out quite yet

“…The approach assumes that the endpoint is easily identified, objectified, and measured. 27 However, our study shows that the range of important contributions that education can make to successful change in practice can be diverse, difficult to anticipate, and separated from the educational intervention by months or even years.…”

Getting Off the “Gold Standard”: Randomized Controlled Trials and Education Research

“…When should an RCT be used in education experiments? According to Norman, 6 randomization is most useful in examining relatively standardized interventions, such as web-based learning and, possibly, clinical simulation. He recommends that randomization be considered when (1) prior observational studies support the hypothesis;…”

“…3,4 The highly complex system of education may be a poor fit for the RCT model, which requires clear inclusion/exclusion criteria and interventions administered identically via multiple physicians (ie, teachers). 3,5,6 Regehr asks whether simulating placebocontrolled efficacy clinical trials, in which 1 or a few variables may be tightly controlled, is a worthwhile goal for medical education research. 3 In education studies, variables can rarely be controlled tightly and blinding of subjects and study personnel may be unethical or impossible.…”