A meta-analysis of the effectiveness of intelligent tutoring systems on college students’ academic learning.

Abstract

This meta-analysis synthesizes research on the effectiveness of intelligent tutoring systems (ITS) for college students. Thirty-five reports were found containing 39 studies assessing the effectiveness of 22 types of ITS in higher education settings. Most frequently studied were AutoTutor, Assessment and Learning in Knowledge Spaces, extended Tutor-Expert System, and Web Interface for Statistics Education. Major findings include (a) Overall, ITS had a moderate positive effect on college students' academic learning [g = .32 to g = .37); (b) ITS were less effective than human tutoring, but they outperformed all other instruction methods and learning activities, including traditional classroom instruction, reading printed text or computerized materials, computer-assisted instruction, laboratory or homework assignments, and no-treatment control; (c) ITS's effectiveness did not significantly differ by different ITS, subject domain, or the manner or degree of their involvement in instruction and learning; and (d) effectiveness in earlier studies appeared to be significantly greater than that in more recent studies. In addition, there is some evidence suggesting the importance of teachers and pedagogy in ITS-assisted learning.

Effectiveness of Computer-Based Scaffolding in the Context of Problem-Based Learning for Stem Education: Bayesian Meta-analysis

“…In addition, in the case of conventional instruction, as opposed to problem-centered instructional models, meta-analysis indicated that computer-based scaffolding including intelligent tutoring systems positively impacted students' learning (g = 0.66) regardless of instructor's effects, study types, and region (Kulik and Fletcher 2016). Other meta-analyses on the effectiveness of intelligent tutoring systems showed a wide range of effect sizes: g = 0.41 among college students (Steenbergen-Hu and Cooper 2014), g = 0.09 for K-12 students' mathematical learning (Steenbergen-Hu and Cooper 2013), d = 0.76 when compared the effectiveness with humantutors (VanLehn 2011), and d = 1.00 for scaffolding within an early model of intelligent tutoring systems (Anderson et al 1995). However, no meta-analyses have investigated the effectiveness of computer-based scaffolding in the context of problem-based learning.…”

Treinamento da Habilidade Metacognitiva de Monitoramento do Conhecimento em Sistemas Tutores

“…O trabalho de meta-análise de Ma e colegas [65] não encontrou diferenças significativas no desempenho dos alunos na aprendizagem com STIs e a aprendizagem através de tutoria humana, quando o sistema realiza uma seleção individualizada de problemas, prompts e feedbacks que são entregues ao aprendiz. Por fim, os estudos de meta-análise descritos em [58,59] também encontraram que os STIs podem ser tão efetivos quanto um professor particular.…”

Assessing the Student's Knowledge in Informatics Discipline Using the METEOR Metric

“…Another meta-analysis, more recent (Steenbergen-Hu and Cooper, 2014), "synthesizes research on the effectiveness of intelligent tutoring systems (ITS) for college students" (p.331). The analysis has been carried out based on 39 studies performed on 22 different ITSs used within higher education programs (Steenbergen-Hu and Cooper, 2014).…”