Theory-Based Evaluation of Instruction: Implications for Improving Student Learning Achievement in Postsecondary Education

Evaluating leadership development using the Most Significant Change technique

“…This is because conventional evaluation of university courses tends to rely on standardized surveys which require learners to indicate their level of satisfaction with sets of items based on the learning content, aspects of teaching, and method of delivery, although such surveys may sometimes be supplemented with other sources of data gathered by individual facilitators. Cohen (1981), Denson, Loveday and Dalton (2010), Feldman (1989), Frick, Chadha, Watson, Wang and Green (2008a, 2008b, Kulik (2001), Spiel, Schober and Reimann (2006), Shortland andMark (1987), andWagner (1999) have highlighted the limitations of current approaches and strongly recommend extending data collection beyond student ratings to a more systematic approach to evaluation designs that offer greater understanding of how the learning is translated into the sociocultural contexts of learners' personal and work lives. Learning is essentially a sociocultural activity, where knowledge requirements are shaped by local "rules, values, attitudes, expectations etc" (Ellstrom, Svesson and Aberg, 2004, p. 479) and the material, social, discursive and historical conditions and relations of work contexts (Kemmis, 2005), are well documented.…”

Improving course evaluations to improve instruction and complex learning in higher education

“…Based on course ratings, if students agreed or strongly agreed that instructors used First Principles of Instruction and those students also agreed or strongly agreed that they were engaged successfully in course activities (ALT), then they were much more likely to: 1) report mastery of course objectives, 2) agree that they learned a lot (made learning progress), 3) agree that they were satisfied, and 4) agree that the course and instructor were outstanding. In a somewhat larger study of 190 students in 111 different courses, Frick et al (2008) found similar patterns among TALQ scales derived from student ratings.…”

MOOCs for Research: The Case of the Indiana University Plagiarism Tutorials and Tests

“…MOO-TALQ scales were adapted from those from the teaching and learning quality (TALQ) survey (Frick et al 2009(Frick et al , 2010a. They constructed a course evaluation instrument that provides for student ratings of First Principles of Instruction and Academic Learning Time (ALT).…”

“…The original TALQ consists of 40 items with nine a priori student rating scales: Academic Learning Time, learning progress, student satisfaction, global quality, authentic problems, activation, demonstration, application, and integration (Frick et al 2010a). In comparison, the MOO-TALQ was designed to evaluate learning experiences in massively open online contexts (what Spector 2014, referred to as mini-MOOCs).…”