Assessing Individual Change Using Short Tests and Questionnaires

Abstract

In clinical contexts, tests and questionnaires are used to assess change at the level of the individual client. The difference between an individual client's posttreatment and pretreatment scores is used to decide about the degree to which the client benefited from a treatment. Because administration time is limited, clinicians prefer using short tests consisting of, say, at the most 15 items. Simulation research showed that shorter tests produce a higher risk of drawing incorrect conclusions about change in individual clients. Based on the simulation results, the authors provide guidelines for the number of items minimally required to reliably assess individual change. Using item response theory, they also provide guidelines for the minimally required local test information for individual-change assessment.

Methods matter: Testing competing models for designing short-scale Big-Five assessments

“…Recent publications on shortened measures recommend caution when using shortened scales, particularly when the length is reduced to one or two items per construct, because of the loss in measurement precision and the increase in Type 1 and Type 2 error (e.g., Credé, Harms, Niehorster, & Gaye-Valentine, 2012; Kruyen, Emons, & Sijtsma, 2012, 2013a, 2013bSchipolowski, Schroeders, & Wilhelm, 2014).…”

Time-Efficient Adaptive Measurement of Change

“…In sum, the findings suggest the utility of the Fisher information per time unit item selection procedure in AMC contexts in which greater TPR to detect change is desired and time is of the essence. In the fields of medicine and psychology, the importance of the measurement of individual change has been well documented (Brouwer et al, 2013;Hays et al, 2019;Hung et al, 2019;Kashikar-Zuck et al, 2016;Kruyen et al, 2014;Wang & Weiss, 2018), and efficiency might be particularly critical when assessment is performed more than once, given the limited time of respondents and providers (Kruyen et al, 2014;Smits et al, 2012). In education, the determination of a given student's improvement, decline, or stasis over time is fundamental information for the student as well as for parents and teachers (Wang & Weiss, 2018;Weiss & Von Minden, 2011); and the efficiency of assessment is again crucial when, for example, testing is conducted in the classroom setting.…”

“…Indeed, the measurement of individual change has been studied among chiropractic patients with chronic neck or low back pain (Hays, Spritzer, Sherbourne, Ryan, & Coulter, 2019), patients undergoing foot and ankle treatment (Hung et al, 2019), and children with chronic pain (Kashikar-Zuck et al, 2016). Psychological assessment may also be conducted on more than one occasion for a given respondent to determine whether a significant change in their beliefs, mental health, or behavior has occurred (Brouwer, Meijer, & Zevalkink, 2013;Kruyen, Emons, & Sijtsma, 2014;Wang & Weiss, 2018). In the field of education, testing might be repeated to gauge a student's degree of improvement (or decline) in a given domain over time (Wang & Weiss, 2018;Weiss & Von Minden, 2011).…”

“…Administrative burden refers to the extent to which costs are incurred through the supervision of the assessment process (Forbey & Ben-Porath, 2007). When assessment is conducted on multiple occasions, it is vital to keep instruments short in order to avoid requiring an inordinate amount of time from respondents and providers (Kruyen et al, 2014;Smits, Zitman, Cuijpers, den Hollander-Gijsman, & Carlier, 2012). The aforementioned benefit of CAT in improving the efficiency of measurement thus becomes particularly important for assessments that are repeated longitudinally.…”

Identifying key domains of health-related quality of life for patients with chronic obstructive pulmonary disease: interviews with healthcare professionals

“…Although PROMs hold a lot of promise to help HCPs in assessing HRQoL in a quick and standardized way, picking and choosing from these instruments can be a daunting task. When selecting a questionnaire for a particular application, aspects to consider include domain coverage, test length (e.g., [11,12]), psychometric properties (e.g., [13]) and whether it is generic or disease specific, among other things [14]. Since both generic and disease-specific instruments have desirable properties, several researchers advocate using both types of instruments to assess HRQoL in patients with COPD [15,16]; however, HCPs tend to prefer short instruments for use in clinical practice and may therefore not heed these recommendations.…”