Effects on Test Performance and Efficiency of Technology-Enhanced Items: An Analysis of Drag-and-Drop Response Interactions

Abstract

One of the most common technology-enhanced items used in large-scale K-12 testing programs is the drag-and-drop response interaction. The main research questions in this study are: (a) Does adding a drag-and-drop interface to an online test affect the accuracy of student performance? (b) Does adding a drag-and-drop interface to an online test affect the speed of student performance? In three different experiments involving fourth, sixth, and eighth graders, respectively; students answered reading comprehension questions presented in conventional (i.e., paper-based design) or drag-and-drop formats. The tests consisted of four-sentence ordering items in Experiment 1, four graphic organizer items in Experiment 2, and two cloze tests and two graphic organizer items in Experiment 3. The conventional and drag & drop groups were compared on test performance (i.e., accuracy) and efficiency (i.e., response time and number of mouse clicks). Across the three experiments, the conventional and drag & drop groups did not differ in mean performance, but the drag & drop group responded more efficiently than the conventional group (faster response time, d = 0.62, and fewer mouse clicks, d = 1.13).

Network Intelligent Education System Based on the Deep Learning Algorithm

“…is not only reduces the time spent in feature engineering but also can automatically combine according to data features, which e entire process of automated AI processing, from data importing to model training, can be done through drag-and-drop interactive interfaces [14]. is ease of use e ectively expands the application scenarios and scope of arti cial intelligence in education, enables ordinary educators to use arti cial intelligence technology to solve educational problems in reality, and provides an e ective way for people who lack professional technical knowledge to quickly build arti cial intelligence applications.…”

Measurement Efficiency for Technology‐Enhanced and Multiple‐Choice Items in a K–12 Mathematics Accountability Assessment

“…Another reason for observing varying degrees of measurement efficiency may be related to computer and human interaction. For instance, previous researchers showed that different layout variants of an item (e.g., presenting resources first vs. presenting targets first as in a GGM item) or adding interaction to a conventional item, can affect the test taker's use of time to respond to an item; on the other hand, it may not affect the response accuracy (Arslan et al., 2020; Moon et al., 2019; Ponce et al., 2021).…”

“…Examples of item interactivity include building and running a simulation for a K-12 science item (Minnesota Department of Education, n.d.), or clicking tabs to access reference materials that mimics literature/document reviewing (American Institute of Certified Public Accountants [AICPA, n.d.]). Interactivity is an important aspect of TE items to reduce cognitive load on test taker working memory and allows more efficient responses to items (Ponce et al, 2021). Media inclusion such as graphics, video, or sound clips may be added to the item stem or response options.…”

The use of process data in large-scale assessments: a literature review

“…These studies looked at innovative item designs or layouts and how they differ from traditional multiple-choice items, mainly in terms of response time. For example, they tested the effects of innovative designs on response accuracy and speed with different results depending on the item type (Ponce et al, 2021;Saß et al, 2012). Other studies compared the information and efficiency (information per time unit) of these items, finding that they are at least as informative as multiple-choice items (Jodoin, 2003;Ponce et al, 2020;Wan & Henly, 2012).…”