Measuring Learning Effectiveness: A New Look at No-Significant-Difference Findings

Abstract

Researchers, instructional designers and consumers of ALNs must be cautious when interpreting results of media comparison studies. Much of the literature purports to have found no significant difference in learning effectiveness between technology-based and conventional delivery media. This research, though, is largely flawed. In this paper, we first outline the philosophical positions of the opposing sides of an intense debate in the literature as to whether delivery media alone influence learning outcomes. We then select at random several representative media comparison studies to illustrate the inadequacy of their methodologies and conclusions. More important, we derive critical design considerations for those who evaluate or conduct media comparison research. ALN practitioners should not assume that students would learn better from technology delivery systems. Rather, ALN practitioners should adhere to time-tested instructional design strategies, regardless of the medium they choose. Learning effectiveness is a function of effective pedagogical practices. Accordingly, the question for ALN practitioners ought to be: "What combination of instructional strategies and delivery media will best produce the desired learning outcome for the intended audience?"

Online Engineering Education: Learning Anywhere, Anytime

“…Providing self-paced modules to students allows additional time for participants in instructor-led courses to engage in interactive exercises. The use of self-paced modules and modules that reduce the amount instructor time for preparation can be found on sites such as Merlot [81] and NEEDS [82]. Experiments are currently in progress that could provide significant benefits to technology-based learning that might become part of a more complete online education package.…”

“…Cost reductions can occur through providing excellent simulations, learning materials, and instructor guides that are used by many students. Many simulations already exist (e.g., the capstone business simulation [79]); a significant number of useful simulation materials are shared among faculty at such Web sites as Merlot [81] or NEEDS [82]. Reinventing ways to present materials, organize topics, build simulations, and test takes remarkable amounts of instructor time.…”

Asynchronous Discussion Groups in Teacher Training Classes: Perceptions of Native and Non-Native Students

“…Asynchronous discussions have been found to promote such development of critical thinking [7,16,17,18,19,20] reflected in "a higher level of scholarly discourse than is typical in many [face-to-face] classes" [14]. Joy and Garcia claim that "learning effectiveness is a function of effective pedagogical practices" and urge that research should focus on "what combination of instructional strategies and delivery media will best produce the desired learning outcome for the intended audience" [8].…”

The effects of instructor immediacy behaviours on student learning experience in an online course

“…Students can complete online courses while working or in their own home (Ally, 2004). No significant differences have been found in learning between students taking online courses and students in traditional classroom settings (Joy and Garcia, 2000;LaRose, Gregg and Eastin, 1998).…”

“…There is a large body of literature examining the use of technologies in online courses (see for example, Benbunan-Fich and Hiltz, 1999;Joy and Garcia, 2000).…”