The Incidence of “Causal” Statements in Teaching-and-Learning Research Journals

Abstract

The authors examined the methodologies of articles in teaching-and-learning research journals, published in 1994 and in 2004, and classified them as either intervention (based on researcher-manipulated variables) or nonintervention. Consistent with the findings of Hsieh et al., intervention research articles declined from 45% in 1994 to 33% in 2004. For nonintervention articles, the authors recorded the incidence of “causal” statements (e.g., if teachers/schools/parents did X, then student/child outcome Y would likely result). Nonintervention research articles containing causal statements increased from 34% in 1994 to 43% in 2004. It appears that at the same time intervention studies are becoming less prevalent in the teaching-and-learning research literature, researchers are more inclined to include causal statements in nonintervention studies.

A Quasi-Experimental Evaluation of An On-Line Formative Assessment and Tutoring System

“…Interestingly, Robinson, Levin, Thomas, and Vaughn (2007) report an increase in causal statements in teaching-and-learning journals from 1994 to 2004, even though during this same time period there was a decline in intervention studies. While the gold-standard for intervention studies is the controlled randomized experiment, the feasibility of running a true experiment in the field is often prohibited by practical issues of compliance, cost-effectiveness, and the ethics of withholding a potentially positive intervention.…”

Research Design, as Independent of Methods

“…I remember clearly being told by international experts that triangulation was based on having three points of view, or that the finite population correction meant that a sample must be smaller than proposed, for example. I have heard colleagues co-teaching in my own modules tell our students that regression is a test of causation (see also Robinson, et al 2007), or that software like Nvivo will analyse textual data for them. Some examples are more serious.…”

The Not-So-Quiet Revolution: Cautionary Comments on the Rejection of Hypothesis Testing in Favor of a “Causal” Modeling Alternative

“…In another journal survey (Robinson et al, 2007), the proportion of articles based on intervention methods had decreased from 45% in 1994 to 33% in 2004. Meanwhile, the proportion of nonintervention articles that contained prescriptive statements increased from 34% in 1994 to 43% in 2004.…”

“…To avoid redundancy, we offer only two such unjustified causal excerpts here, from numerous ones that we have encountered in multiple teaching-and-learning research journals that we have recently read or reviewed (see Robinson, Levin, Thomas, Pituch, & Vaughn, 2007, and the following section).…”