Modeling Response Time and Responses in Multidimensional Health Measurement

Abstract

This study explored calibrating a large item bank for use in multidimensional health measurement with computerized adaptive testing, using both item responses and response time (RT) information. The Activity Measure for Post-Acute Care is a patient-reported outcomes measure comprised of three correlated scales (Applied Cognition, Daily Activities, and Mobility). All items from each scale are Likert type, so that a respondent chooses a response from an ordered set of four response options. The most appropriate item response theory model for analyzing and scoring these items is the multidimensional graded response model (MGRM). During the field testing of the items, an interviewer read each item to a patient and recorded, on a tablet computer, the patient's responses and the software recorded RTs. Due to the large item bank with over 300 items, data collection was conducted in four batches with a common set of anchor items to link the scale. van der Linden's (2007) hierarchical modeling framework was adopted. Several models, with or without interviewer as a covariate and with or without interaction between interviewer and items, were compared for each batch of data. It was found that the model with the interaction between interviewer and item, when the interaction effect was constrained to be proportional, fit the data best. Therefore, the final hierarchical model with a lognormal model for RT and the MGRM for response data was fitted to all batches of data via a concurrent calibration. Evaluation of parameter estimates revealed that (1) adding response time information did not affect the item parameter estimates and their standard errors significantly; (2) adding response time information helped reduce the standard error of patients' multidimensional latent trait estimates, but adding interviewer as a covariate did not result in further improvement. Implications of the findings for follow up adaptive test delivery design are discussed.

Prediction of Mental Health Problem Using Annual Student Health Survey: Machine Learning Approach

“…We use the word answering time as a broad concept, including response time (RT), which is the amount of time in which a student responds to the question item after it is shown on the screen; RT stamp (the time of day when a student accesses the survey form); and answer date (the number of days elapsed in which a student completes the survey after it was announced). In particular, RT has recently attracted attention mainly in psychological assessment and educational testing; however, little is known about how to use RT in mental health measurement [ 25 , 26 ]. A few studies have shown that RT is related to depression estimation and the prediction of suicide ideation and attempt status [ 25 , 27 ].…”

Using Machine Learning Methods to Develop a Short Tree-Based Adaptive Classification Test: Case Study With a High-Dimensional Item Pool and Imbalanced Data

“…CAT based on multidimensional IRT models has also been proposed to efficiently measure multiple traits at the same time (e.g., Seo & Weiss, 2015; Wang et al, 2019; Yao et al, 2014; Zheng et al, 2013). Such CAT algorithms based on multidimensional IRT models are statistically and computationally intense.…”

LASSO-Based Pattern Recognition for Replenished Items With Graded Responses in Multidimensional Computerized Adaptive Testing

“…The research scenario of LPRM can be expanded, such as establishing a pattern recognition framework for the MCAT with polytomous items. Note that for the multidimensional graded-response data, the MGRM is an essential compensatory MIRT model for the paper-and-pencil test or the MCAT (e.g., Jiang et al, 2016;Depaoli et al, 2018;Tu et al, 2018;Wang et al, 2018a, Wang et al, 2019Nouri et al, 2021), so it is necessary to extend the item-trait pattern recognition idea for the items with dichotomous responses to those with graded responses. Moreover, the LPRM was developed directly based on the M2PLM.…”