Longitudinal Analysis of Patient-Reported Outcomes in Clinical Trials: Applications of Multilevel and Multidimensional Item Response Theory

Cai, Li; Houts, Carrie R.

doi:10.1007/s11336-021-09777-y

Cited by 24 publications

(21 citation statements)

References 36 publications

(62 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For the P method, expected a posteriori (EAP) estimates for θ were used for the computation. Item calibration and EAP scoring were conducted using flexMIRT (Cai, 2017).…”

Section: Estimation Of Classification Consistency and Accuracymentioning

confidence: 99%

Estimating Classification Accuracy and Consistency Indices for Multiple Measures with the Simple Structure MIRT Model

Park

Kim

Lee

2022

J Educational Measurement

View full text Add to dashboard Cite

Multiple measures, such as multiple content domains or multiple types of performance, are used in various testing programs to classify examinees for screening or selection. Despite the popular usages of multiple measures, there is little research on classification consistency and accuracy of multiple measures. Accordingly, this study introduces an approach to estimate classification consistency and accuracy indices for multiple measures under four possible decision rules: (1) complementary, (2) conjunctive, (3) compensatory, and (4) pairwise combinations of the three. The current study uses the IRT-recursive-based approach with the simple-structure multidimensional IRT model (SS-MIRT) to estimate the classification consistency and accuracy for multiple measures. Theoretical formulations of the four decision rules with a binary decision (Pass/Fail) are presented. The estimation procedures are illustrated using an empirical data example based on SS-MIRT. In addition, this study applies the estimation procedures to the unidimensional IRT (UIRT) context, considering that UIRT is practically used more. This application shows that the proposed procedure of classification consistency and accuracy could be used with a UIRT model for individual measures as an alternative method of SS-MIRT.Classification based on educational and psychosocial assessments directly impacts examinees' future learning (e.g., high school equivalency credential) or career success (e.g., certificate or licensure examination). An inaccurate classification would cause severe negative consequences. For instance, students who are incorrectly classified as "Fail" might not be able to have access to higher education. In addition, if a medical certificate is issued to a candidate who is not qualified, that candidate is more likely to cause medical accidents. Thus, classifying examinees accurately is important.Many high-stakes assessments involve the use of multiple measures to classify examinees; that is, multiple content domains or multiple types of performances are used to measure required abilities and skills accurately (Baker, 2003;Brookhart, 2009;Douglas & Mislevy, 2010;Henderson-Montero et al., 2003). The HiSET® exam (Educational Testing Service, 2018), which is a high school equivalency credential exam, is an example that consists of multiple domains-language arts-reading, language art-writing, mathematics, science, and social studies. Multiple types of performances are also commonly used in classification contexts. The Uniform Bar Examination

show abstract

“…For the P method, expected a posteriori (EAP) estimates for θ were used for the computation. Item calibration and EAP scoring were conducted using flexMIRT (Cai, 2017).…”

Section: Estimation Of Classification Consistency and Accuracymentioning

confidence: 99%

Estimating Classification Accuracy and Consistency Indices for Multiple Measures with the Simple Structure MIRT Model

Park

Kim

Lee

2022

J Educational Measurement

View full text Add to dashboard Cite

show abstract

“…The response data were then calibrated to obtain item parameter estimates for the pool. Calibration was implemented using the MMLE-EM procedure with flexMIRT (Cai, 2017). The maximum number of cycles was 500; the convergence criterion was 1.00e-004; and there were 54 quadrature points between -4.0 and 4.0.…”

Section: Item Poolmentioning

confidence: 99%

“…Implementation of SC and FC resulted in differences and commonalities. Both SC and FC were implemented using the MMLE-EM procedure via flexMIRT (Cai, 2017). MMLE-EM control values used for calibrating the item pool were common to all calibration approaches.…”

Section: Simulationmentioning

confidence: 99%

Pretest Item Calibration in Computerized Multistage Adaptive Testing

Erşen

Lee

2023

J Educational Measurement

View full text Add to dashboard Cite

The purpose of this study was to compare calibration and linking methods for placing pretest item parameter estimates on the item pool scale in a 1‐3 computerized multistage adaptive testing design in terms of item parameter recovery. Two models were used: embedded‐section, in which pretest items were administered within a separate module, and embedded‐items, in which pretest items were distributed across operational modules. The calibration methods were separate calibration with linking (SC) and fixed calibration (FC) with three parallel approaches under each (FC‐1 and SC‐1; FC‐2 and SC‐2; and FC‐3 and SC‐3). The FC‐1 and SC‐1 used only operational items in the routing module to link pretest items. The FC‐2 and SC‐2 also used only operational items in the routing module for linking, but in addition, the operational items in second stage modules were freely estimated. The FC‐3 and SC‐3 used operational items in all modules to link pretest items. The third calibration approach (i.e., FC‐3 and SC‐3) yielded the best results. For all three approaches, SC outperformed FC in all study conditions which were module length, sample size and examinee distributions.

show abstract

“…Within each of the latent variable vectors ϑj and η ij , latent variables may be correlated, or a bifactor or testlet structure could be specified at each level. As with the single-level IRT models, MLIRT models can be expanded to include both a measurement model and a conditioning model (Cai & Houts, 2021).…”

Section: Approaches To Producing Scores In a Multisite Cluster Rctmentioning

confidence: 99%

Scoring assessments in multisite randomized control trials: Examining the sensitivity of treatment effect estimates to measurement choices.

Kuhfeld,

Soland

2023

Psychological Methods

View full text Add to dashboard Cite

Longitudinal Analysis of Patient-Reported Outcomes in Clinical Trials: Applications of Multilevel and Multidimensional Item Response Theory

Cited by 24 publications

References 36 publications

Estimating Classification Accuracy and Consistency Indices for Multiple Measures with the Simple Structure MIRT Model

Estimating Classification Accuracy and Consistency Indices for Multiple Measures with the Simple Structure MIRT Model

Pretest Item Calibration in Computerized Multistage Adaptive Testing

Scoring assessments in multisite randomized control trials: Examining the sensitivity of treatment effect estimates to measurement choices.

Contact Info

Product

Resources

About