Test takers' experiences during adaptive versus fixed item testing

Abstract

of the College Board, graces this issue's cover. This richly detailed visual depicts examinees' reading abilities across the multiple dimensions of a large-scale measure of reading ability. These creative researchers described their work this way. This visual answers the following question: "Based on ex-aminees' performance on 40+ reading items, is it possible to visually identify clusters of examinees that have different reading skills?" The plot answers the question by showing three panels: the outer two visualize different views of a cluster analysis , and the middle panel visualizes differences in skills for the identified clusters. The plots show test takers' strengths and weaknesses in their reading abilities for a large-scale assessment. The outer panels show different views of low-dimensional clustering of n = 4,930 test takers' performance on 40+ reading items, and the middle panel represents reading skill profiles of eight clusters of test takers. The outer panels are visual representations of n = 4,930 test takers' performances on reading items of a large-scale assessment. With more than 40 items, it is typically difficult to visualize test takers' individual performances, especially when attempting to identify differences in subgroups of test takers by skill. Using an innovative approach, the researchers first reduced the data into three dimensions using T-distributed stochastic neighbor embedding (T-SNE), a relatively new machine learning technique that facilitates visualization of high-dimensional data in low dimensions. Then they applied model-based cluster analysis based on Gaussian mixture mod-eling, and classified all test takers into eight clusters based on item-level performance. Finally, they generated an interactive 3D plot using the R plotly package (R Core Team, 2016). The outer panels contain four different angles of the 3D plot. The middle panel shows the skill profiles of the eight clusters. Content experts theorized three primary skills for reading, and then identified which skill is primarily measured by each item. We calculated each cluster's mean proportion of correct responses for each skill and then plotted the cluster profiles. The eight profiles have various patterns, but the major difference lies in the elevation/level, which indicates that the three skills are highly correlated and compensatory. The visualization confirmed that the total reading score was a sufficient metric for understanding examinee achievement, and that clusters of similar-performing students were mainly differentiated by their "level" of performance and not by individual strengths or weaknesses in particular skills.

Achievement Goal Impacts Students' Preferences for “Personalized Problems” in Computer‐Adaptive Tests

“…Some studies have reported that highly anxious students are more likely to perform poorly, even when they work on computer-adaptive tests (Lu et al, 2016;Ortner & Caspers, 2011). Moreover, some found that students felt more anxious when taking a computer-adaptive test than when taking a test in the traditional format (Ortner et al, 2014). Students may feel pressured by personalized problems, which require constant challenge, as they experience almost equal amounts of successes and failures during tests.…”