Measurement Qualities of a Self-Report and Therapist-Scored Functional Capacity Instrument Based on the Dictionary of Occupational Titles

Abstract

Studies provide convincing arguments to support the development of functional capacity instruments based on the Dictionary of Occupational Titles (DOT). The purpose of this study is to investigate the item-level measurement qualities of a newly developed DOT-based functional capacity instrument for clients undergoing rehabilitation treatment for back pain. Client and therapist ratings were collected on 124 clients from 27 rehabilitation sites using the newly developed Occupational Rehabilitation Data Base (ORDB) functional capacity instrument. Rasch analysis was used to investigate: (1) unidimensionality, (2) hierarchical item difficulty continuum, (3) rater severity, and 4) person-item match. Overall, the functional capacity scale of the ORDB showed good measurement qualities. All items, except the Handling item fit the Rasch measurement model. Because of high fit statistics and loading on factors independent from the remainder of the items, the "handling" item was removed, from further analyses. Separate client-rated and therapist-rated instruments retained good item-level psychometrics. While client and therapist items showed similar item-difficulty hierarchical structures, clients had a tendency to be more severe in their rating and the correlation between client and therapist ratings was relatively low, 0.32. These findings suggest that Handling items should not be included as a DOT measure for clients with back pain. While the above psychometric study supports using client or therapist ratings as independent instruments, the lack of concordance between these ratings requires further investigation.

Computer Adaptive Testing Method for Measuring Disability in Patients With Back Pain

“…In contrast to the CTT, the IRT focuses on the psychometric properties of the items making up instrument instead of the instrument as a whole (Velozo et al, 2006). By estimating the probability that a respondent will select a particular rating for an item, item difficulty and person ability (or disability) can be placed on the same linear continuum.…”

A Comparative Study of Oswestry Back Pain Disability Questionnaire Versus Computer Adaptive Testing for Measuring Back Pain

“…In contrast to CTT, item response theory (IRT) focuses on the psychometric properties of the items making up the instrument instead of the instrument as a whole (Velozo et al, 2006;Velozo et al, 2008). By estimating the probability that a respondent will select a particular rating for an item, item difficulty and person ability can be placed on the same linear continuum.…”

“…By estimating the probability that a respondent will select a particular rating for an item, item difficulty and person ability can be placed on the same linear continuum. Thus, IRT model allows "connecting" individuals' responses to items with their ability level (Velozo and Peterson, 2001;Velozo et al, 2006). Estimates of person ability on an underlying construct obtained using IRT methods are invariant regardless of the items used, whereas under the CTT paradigm, person scores vary depending on the difficulty of the instrument (Velozo et al, 2006).…”

“…Thus, IRT model allows "connecting" individuals' responses to items with their ability level (Velozo and Peterson, 2001;Velozo et al, 2006). Estimates of person ability on an underlying construct obtained using IRT methods are invariant regardless of the items used, whereas under the CTT paradigm, person scores vary depending on the difficulty of the instrument (Velozo et al, 2006). Furthermore, item difficulty estimates derived from the IRT analyses remain the same regardless of the ability of the sample, while test statistics in CTT are dependent on the sample taking the test.…”

“…Studies have shown that CAT improves test efficiency maintaining adequate precision with fewer items than the full test (Elhan et al, 2008;Flynn et al, 2008;Haley et al, 2008;Hart et al, 2006;Hol et al, 2007;Jette et al, 2008;Velozo et al, 2006;Ware et al, 2003;Weiss, 1982). CAT measures are highly correlated with other assessments intending to measure the same construct and require fewer items.…”