Automated Test Assembly for Multistage Testing With Cognitive Diagnosis

Abstract

Computer multistage adaptive test (MST) combines the advantages of paper and pencil-based test (P&P) and computer-adaptive test (CAT). As CAT, MST is adaptive based on modules; as P&P, MST can meet the need of test developers to manage test forms and keep test forms parallel. Cognitive diagnosis (CD) can accurately measure students’ knowledge states (KSs) and provide diagnostic information, which is conducive to student’s self-learning and teacher’s targeted teaching. Although MST and CD have a lot of advantages, many factors prevent MST from applying to CD. In this study, we first attempt to employ automated test assembly (ATA) to achieve the objectives of MST in the application of CD (called CD-MST) via heuristic algorithms. The mean correct response probability of all KSs for each item is used to describe the item difficulty of CD. The attribute reliability in CD is defined as the test quantitative target. A simulation study with the G-DINA model (generalized deterministic input noisy “and” gate model) was carried out to investigate the proposed CD-MST, and the results showed that the assembled panels of CD-MST satisfied the statistical and the non-statistical constraints.

The Use of Stocking-Lord and Haebara Methods in Horizontal Equating: A Case of Indonesian Madrasah Competence Assessment

“…MSAT is an exam administration method that has recently become popular in assessment (Cai et al, 2021;Li et al, 2021;MacGregor et al, 2022;Shin et al, 2021). Test administration using the MSAT model can increase measurement efficiency (Berger et al, 2019).…”

Sociocultural Literacy Assessment: Validation of Multistage Generalized Partial Credit Testing Design

“…Multistage testing (MST) is a popular approach of administering tests [1][2][3][4]. Its effectivity in accommodating the balance between test content and security attract test administrators to adapt it.…”