Application of Network Analysis to Description and Prediction of Assessment Outcomes

Abstract

With the use of computerized testing, ordinary assessments can capture both answer accuracy and answer response time. For the Canadian Programme for the International Assessment of Adult Competencies (PIAAC) numeracy and literacy subtests, person ability, person speed, question difficulty, question time intensity, fluency (rate), person fluency (skill), question fluency (load), pace (rank of response time within question), and person pace were assessed. Undirected Gaussian Graphical Model networks of the measures based on partial correlations were predictive of the measures as nodes. The population-based model extrapolated well to individual person estimations. Finally, it was shown that the "training" Canadian model generalized with minor differences to four other English-speaking PIAAC assessments (USA , Great Britain, Ireland, and New Zealand). Thus, the undirected network approach provides a heuristic that is both descriptive and predictive. However, the model is not causal and can be taken as an example of "mutualism".

Pace or Inverse Error Provide Facile Observational-Level Approximations to Logarithmic Response Time Residuals and Response Time in the Analysis of Computerized Assessments

“…Empirical Application. Previously described USA PIAAC data is analyzed here (Thompson, 2022). It is comprised of binary scores and response times from 1806 persons.…”

“…A method (Thompson, 2022) that both describes and predicts models such as those listed above has been proposed (undirected network analysis). In this paradigm, variables are considered nodes, and partial correlations between variables are depicted as edges.…”

“…When used for inverse error calculation, pace[0,1] was rescaled to pace[0.000001,0.99999] with the scales package. Empirical response time data used for comparative analyses was the USA PIAAC data used by Thompson (Thompson, 2022). Person ability (ability), person speed (speed), question difficulty (QD), question time intensity (QTI), score discrimination (PMID) and time discrimination (PMTD) were calculated from the LNIRT package.…”