Why item response theory should be used for longitudinal questionnaire data analysis in medical research

Abstract

BackgroundMulti-item questionnaires are important instruments for monitoring health in epidemiological longitudinal studies. Mostly sum-scores are used as a summary measure for these multi-item questionnaires. The objective of this study was to show the negative impact of using sum-score based longitudinal data analysis instead of Item Response Theory (IRT)-based plausible values.MethodsIn a simulation study (varying the number of items, sample size, and distribution of the outcomes) the parameter estimates resulting from both modeling techniques were compared to the true values. Next, the models were applied to an example dataset from the Amsterdam Growth and Health Longitudinal Study (AGHLS).ResultsThe results show that using sum-scores leads to overestimation of the within person (repeated measurement) variance and underestimation of the between person variance.ConclusionsWe recommend using IRT-based plausible value techniques for analyzing repeatedly measured multi-item questionnaire data.Electronic supplementary materialThe online version of this article (doi:10.1186/s12874-015-0050-x) contains supplementary material, which is available to authorized users.

Language‐related differential item functioning between English and German PROMIS Depression items is negligible

“…For this, we used our new web application (http://www.common-metrics.org). We drew 25 sets of plausible values (Gorter et al, ) from the resulting expected a posteriori (EAP) estimates and estimated the mean for each sample in a linear model for each draw. The mean estimates from these 25 models were then combined according to Rubin's rule (Schafer & Graham, ).…”

“…The resulting models were used to estimate theta with the EAP technique in both the German and the US sample. Since direct use of EAP estimates in regression models does not account for uncertainty in theta, we drew 25 samples of plausible values (Gorter et al, ) and fitted three ordered logistic regression models with the following assumptions in each of the 25 imputed datasets: Model 1 (no DIF): item response ~ theta Model 2 (uniform DIF): item response ~ theta + language Model 3 (non‐uniform DIF, interaction term included): item response ~ theta + language + theta*language …”

Treatment-seeking behaviour in low- and middle-income countries estimated using a Bayesian model

“…In education, for example, it has been used to estimate the person-level traits (such as ability) or item-level difficulty in an examination [29–31]. IRT concepts can be extended to health as applied previously in delirium screening [32], longitudinal data analysis [33], and interpreting medical codes from patient records [34]. IRT approaches are essentially probit models with additional regression effects used to aid estimation of item characteristics [35].…”

Implementation and effects of a school-wide data-based decision making intervention

“…However, by using means or sum scores, between-person differences in response patterns leading to the same score are ignored. As a result, respondents with different response patterns obtain the same score, and valuable information about the response patterns is lost (Gorter, Fox, & Twisk, 2015). In this study, multilevel IRT (leading to a continuous score) and multilevel LCA (leading to a categorical score) methods were used instead.…”

“…Furthermore, the mean score summarizes the response-pattern information, and ignores differences between response patterns leading to the same mean score. In longitudinal data analysis, the mean score as an outcome measure can lead to substantial bias in parameter estimates leading to incorrect statistical inferences (Gorter, Fox, & Twisk, 2015).…”