Rasch Analysis as a Technique to Examine the Psychometric Properties of a Career Ability Placement Survey Subtest

Abstract

Rasch analysis offers researchers an innovative method for developing and validating rehabilitation instruments. Despite the increased use of Rasch analysis by allied health researchers, this method continues to be underutilized by rehabilitation counseling professionals. In the rehabilitation counseling profession, vocational evaluators use various instruments, such as the Career Ability Placement Survey (CAPS), to determine client aptitudes and justify job placement recommendations. The purpose of this article is to introduce Rasch analysis as a method for analyzing vocational assessments by examining the psychometric properties of the Word Knowledge (WK) subtest of the CAPS. University undergraduate and graduate students ( n = 227) completed the WK subtest as part of a classroom exercise. The analysis determined item hierarchy, item gaps and redundancies, item-fit statistics (unidimensionality), and person reliability and separation statistics. Although the WK construct demonstrates good psychometric characteristics relating to unidimensionality and to person reliability and separation statistics, several items deviated from the expected ordering pattern. The reordering of items revealed potential item gaps and redundancies as well as poor person—instrument match. Adding and removing items will improve the validity and reliability of the WK subtest psychometrics.

Rasch Analysis of the Lubben Social Network Scale–6 (LSNS-6)

“…Similarly, if there are not enough items toward the bottom, the instrument cannot distinguish among persons with low trait levels. The former is called a ceiling effect, and the latter, a floor effect (Pomeranz, Byers, Moorhouse, Velozo, & Spitznagel, 2008). …”

Examining Psychometric and Measurement Properties of the Career Thoughts Inventory: Demonstration and Use of the Rasch Measurement Model in Career Assessment Research Technical Report No. 51

“…For example, in the current study, the MS cutoff values of .6 to 1.4 were used to identify misfitting items, because Bond and Fox note that these thresholds are appropriate for use with Likert-scaled items. Further, Linacre (2002a) considers items misfitting when they exceed a standardized (ZSTD) value of +/-2.00; and Pomeranz et al (2008) note, "For a researcher to consider an item "misfitting," the item must exceed both [italics added] the MS and ZSTD criteria" (p. 253).…”

The Validity of Standardized Interviews Used for University Admission Into Health Professional Programs

“…Furthermore, the relationship that exists between person ability and item difficulty is able to capture the relationship of the item and person (Coe, 2008;Wilmot, Schoenfeld, Wilson, Champney, & Zahner, 2011). The Rasch model is able to provide a rigorous and detailed methodology to identify the psychometric properties of an instrument at the item level (Pomeranz, Byers, Moorhouse, Velozo, & Spitznagel, 2008;Tran et al, 2010). It has proven to elevate test design to a level of sophistication not otherwise possible when using only raw scores to determine an individual's ability (McAllister, Lincoln, Ferguson, & McAllister, 2010;Potgieter, Davidowitz, & Venter, 2008).…”

“…It has proven to elevate test design to a level of sophistication not otherwise possible when using only raw scores to determine an individual's ability (McAllister, Lincoln, Ferguson, & McAllister, 2010;Potgieter, Davidowitz, & Venter, 2008). The Rasch model is able to provide details of the validity and reliability of instruments by focusing specifically on rating scales, items, persons, and other facets like the rater (Coe, 2008;Pomeranz et al, 2008).…”