Evaluation criteria for artificial intelligence

Abstract

Criteria identify and define the aspects on which what we evaluate is judged and play a central role in evaluation practice. While work on the use of AI in evaluation is burgeoning, at the time of writing, a set of criteria to consider in evaluating the use of AI in evaluation has not been proposed. As a first step in this direction, Teasdale's Criteria Domains Framework was used as the lens through which to critically read articles included in this special issue. This resulted in the identification of eight criteria domains for evaluating the use of AI in evaluation. Three of these criteria domains relate to the conceptualization and implementation of AI in evaluation practice. Five criteria domains are focused on outcomes, specifically those stemming from the use of AI in evaluation. More work is needed to further identify and deliberate possible criteria domains for AI use in evaluation.