Psychometrics of the Fitness-to-Drive Screening Measure

Abstract

We employed item response theory (IRT), specifically using Rasch modeling, to determine the measurement precision of the Fitness-to-Drive Screening Measure (FTDS), a tool that can be used by caregivers and occupational therapists to help detect at-risk drivers. We examined unidimensionality through the factor structure (how items contribute to the central construct of fitness to drive), rating scale (use of the categories of the rating scale), item/person-level separation (distinguishing between items with different difficulty levels or persons with different ability levels) and reliability, item hierarchy (easier driving items advancing to more difficult driving items), rater reliability, rater effects (severity vs. leniency of a rater), and criterion validity of the FTDS to an on-road assessment, via three rater groups (n = 200 older drivers; n = 200 caregivers; n = 2 evaluators). The FTDS is unidimensional, the rating scale performed well, has good person (> 3.07) and item (> 5.43) separation, good person (> 0.90) and item reliability (> 0.97), with < 10% misfitting items for two rater groups (caregivers and drivers). The intraclass correlation (ICC) coefficient among the three rater groups was significant (.253, p < .001) and the evaluators were the most severe raters. When comparing the caregivers' FTDS rating with the drivers' on-road assessment, the areas under the curve (index of discriminability; caregivers .726, p < .001) suggested concurrent validity between the FTDS and the on-road assessment. Despite limitations, the FTDS is a reliable and accurate screening measure for caregivers to help identify at-risk older drivers and for occupational therapy practitioners to start conversations about driving.

Fitness-to-Drive Screening Measure©: Patterns and Trends for Canadian Users

“…Furthermore, rater effects (leniency vs. severity among raters) indicated driving evaluators, as expected, were more severe in their ratings compared to the drivers or proxy raters (Classen, Velozo, et al, 2015). The FTDS also demonstrated concurrent validity with the gold standard, on-road assessment (Classen, Velozo, et al, 2015). Based on cut-points, sensitivity and specificity are established.…”

“…Removing the "pre-driving" (vs. actual driving) items resulted in the existing 54 items fitting a onefactor model for proxy raters and certified driving evaluators. Meeting two of the three criteria for unidimensionality, the FTDS showed good unidimensionality for proxy raters and driving evaluators (Classen, Velozo, et al, 2015). Interrater reliability indicated moderate (ICC = .394,p < .001) correlations between the driving evaluator and proxy raters (Classen, Velozo, et al, 2015;Portney & Watkins, 2008).…”

“…Meeting two of the three criteria for unidimensionality, the FTDS showed good unidimensionality for proxy raters and driving evaluators (Classen, Velozo, et al, 2015). Interrater reliability indicated moderate (ICC = .394,p < .001) correlations between the driving evaluator and proxy raters (Classen, Velozo, et al, 2015;Portney & Watkins, 2008). Furthermore, rater effects (leniency vs. severity among raters) indicated driving evaluators, as expected, were more severe in their ratings compared to the drivers or proxy raters (Classen, Velozo, et al, 2015).…”

“…Studies evaluating the psychometrics of the FTDS have identified the tool as having good face and content validity (Classen, Velozo, et al, 2015).…”

“…Interrater reliability indicated moderate (ICC = .394,p < .001) correlations between the driving evaluator and proxy raters (Classen, Velozo, et al, 2015;Portney & Watkins, 2008). Furthermore, rater effects (leniency vs. severity among raters) indicated driving evaluators, as expected, were more severe in their ratings compared to the drivers or proxy raters (Classen, Velozo, et al, 2015). The FTDS also demonstrated concurrent validity with the gold standard, on-road assessment (Classen, Velozo, et al, 2015).…”