The Assessment of Knowledge, in Theory and in Practice

Abstract

This paper is adapted from a book and many scholarly articles. It reviews the main ideas of a novel theory for the assessment of a student's knowledge in a topic and gives details on a practical implementation in the form of a software system available on the Internet. The basic concept of the theory is the 'knowledge state,' which is the complete set of problems that an individual is capable of solving in a particular topic, such as Arithmetic or Elementary Algebra. The task of the assessor-which is always a computer-consists in uncovering the particular state of the student being assessed, among all the feasible states. Even though the number of knowledge states for a topic may exceed several hundred thousand, these large numbers are well within the capacity of current home or school computers. The result of an assessment consists in two short lists of problems which may be labelled: 'What the student can do' and 'What the student is ready to learn.' In the most important applications of the theory, these two lists specify the exact knowledge state of the individual being assessed. This work is presented against the contrasting background of common methods of assessing human competence through standardized tests providing numerical scores. The philosophy of these methods, and their scientific origin in nineteenth century physics, are briefly examined.

AI as a Methodology for Supporting Educational Praxis and Teacher Metacognition

“…The proposal to create a techno-intellectual infrastructure for supporting educators' metacognition brings to sharp focus several areas of pertinence, namely, academic and industrial attempts to create communities of educational practice such as ASSISTments (Heffernan and Heffernan 2014), ALEKS (Falmagne et al 2006), Learning Designer (Laurillard 2012), or STAR./LBD.Legacy (Schwartz et al 1998); work related to teachers' metacognition and its importance to continuous professional development (CPD) and pedagogical innovation (Lin et al 2005;Lin and Schwartz 2003); and AI, especially knowledge representation and engineering (e.g. Davis et al 1993;Conlon and Pain 1996).…”

“…Some of these examples are inspired or even driven by AI, and there are also applications, whose explicit goal is to contribute to the development of intelligent systems for learning. For instance, ALEKS (Falmagne et al 2006) is AI-driven software for assessment and learning of maths and science from primary to university level, which utilises adaptive questioning, assessment and selection of targeted instruction and feedback to help learners. Knewton (Wilson and Nichols 2015) is also an AI-enabled system (employing advanced ontologies and Bayesian reasoning) for assessing students' fine-grained conceptual progress in maths, English language art, and biology.…”

Evaluation of Automated Diagnosis and Instruction for Mathematics Problem‐solving

“…Could this student nevertheless solve problem p'? Falmagne et al (2004) showed an example from the resulting precedence network where "word problems on proportions" precedes "multiplication of monomials," which in turn precedes "graphing a line through a given point with a given slope." While the two questions imply a direct prerequisite relation from one item type to another, the resulting precedence model seems to be more general than a prerequisite relation, in that, as Falmagne et al pointed out, some concepts are taught in a particular order even though there may be no logical or pedagogical reason to do so (p. 4).…”

“…The authors report that the precedence graphs have been further refined via data from thousands of students (Falmagne et al, 2004). However, no information was found about controlled evaluations using ALEKS as a teaching instrument-from (a) the available/posted research papers on the ALEKS web site, (b) several Google and Google Scholar searches on the topic of evaluations of the system, (c) ERIC and PsychLit searches, or (d) from corresponding by e-mail with the ALEKS information site and the first author directly.…”

Matching learning material to people with disabilities based on competence and functional capability

“…ILO contains learned capability which can be used to evaluate a readiness of learner for further learning rather than assessing knowledge level [3,4]. Each person has different ILO and competences.…”