Using Response Time in Multidimensional Computerized Adaptive Testing

Abstract

In multidimensional computerized adaptive testing (MCAT), item selection strategies are generally constructed based on responses, and they do not consider the response times required by items. This study constructed two new criteria (referred to as DT‐inc and DT) for MCAT item selection by utilizing information from response times. The new designs maximize the amount of information per unit time. Furthermore, these two new designs were extended to the DTS‐inc and DTS designs to efficiently estimate intentional abilities. Moreover, the EAP method for ability estimation was also equipped with response time. The performances of the response‐time‐based EAP (RT‐based EAP) and the new designs were evaluated in simulation and empirical studies. The results showed that the RT‐based EAP significantly improved the ability estimation precision compared with the EAP without using response time, and the new designs dramatically saved testing times for examinees with a small sacrifice of ability estimation precision and item pool usage.

More haste, less speed? Relationship between response time and response accuracy in gamified online quizzes in an undergraduate engineering course

“…Historically, response time has held a central role in cognitive and psychometric research (Thorndike et al, 1926;Gulliksen, 1950;Lohman, 1989). Numerous studies have utilized response time as a proxy to study the cognitive processes underlying test performance (Kyllonen and Thomas, 2020), as an indicator of motivation in computer-based tests (Wise and Kong, 2005), and as a parameter in test design (van der Linden, 2009;Ranger et al, 2020;He and Qi, 2023). Response time can be influenced by various factors, including individual differences in motivation, cognitive capacity, attitudes, personality, question difficulty, and trade-offs related to accuracy (Thurstone, 1937;Lohman, 1989;Ferrando and Lorenzo-Seva, 2007;Ranger and Kuhn, 2012;Partchev et al, 2013;Goldhammer et al, 2014).…”