Computerized Adaptive Testing With Polytomous Items

Dodd, Barbara G.; Ayala, R. J. De; Koch, William R.

doi:10.1177/014662169501900103

Cited by 107 publications

(78 citation statements)

References 45 publications

(57 reference statements)

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…CATs (1) administer informative items, the difficulty of which are matched to the patient's level of ability reducing the number of inappropriate items administered; (2) administer fewer items, reducing respondent burden with little reduction in precision of patient ability estimates; 3) allow the level of measure precision to be established before testing improving control of measurement error during testing; and (4) simplify test revision by allowing adding and testing new items as needed [6,14]. CATs provide an efficient alternative to traditional paper-and-pencil or computer-administered tests, and allow outcomes data to be collected during the clinical encounter with reduced patient and scoring burden.…”

Section: Introductionmentioning

confidence: 99%

Simulated computerized adaptive tests for measuring functional status were efficient with good discriminant validity in patients with hip, knee, or foot/ankle impairments

Hart¹,

Mioduski²,

Stratford

2005

Journal of Clinical Epidemiology

105

112

View full text Add to dashboard Cite

Section: Introductionmentioning

confidence: 99%

Simulated computerized adaptive tests for measuring functional status were efficient with good discriminant validity in patients with hip, knee, or foot/ankle impairments

Hart¹,

Mioduski²,

Stratford

2005

Journal of Clinical Epidemiology

105

112

View full text Add to dashboard Cite

“…In terms of psychiatric measurement, research has demonstrated that both item response theory and CAT (29) can be meaningfully applied to the measurement of attitudes and personality variables (30,31).…”

mentioning

confidence: 99%

Using Computerized Adaptive Testing to Reduce the Burden of Mental Health Assessment

Gibbons

Weiss

Kupfer

et al. 2008

Psychiatric Services

119

View full text Add to dashboard Cite

Objective-This study investigated the combination of item response theory and computerized adaptive testing (CAT) for psychiatric measurement as a means of reducing the burden of research and clinical assessments.Methods-Data were from 800 participants in outpatient treatment for a mood or anxiety disorder; they completed 616 items of the 626-item Mood and Anxiety Spectrum Scales (MASS) at two times. The first administration was used to design and evaluate a CAT version of the MASS by using post hoc simulation. The second confirmed the functioning of CAT in live testing.Results-Tests of competing models based on item response theory supported the scale's bifactor structure, consisting of a primary dimension and four group factors (mood, panic-agoraphobia, obsessive-compulsive, and social phobia). Both simulated and live CAT showed a 95% average reduction (585 items) in items administered (24 and 30 items, respectively) compared with administration of the full MASS. The correlation between scores on the full MASS and the CAT version was .93. For the mood disorder subscale, differences in scores between two groups of depressed patients-one with bipolar disorder and one without-on the full scale and on the CAT showed effect sizes of .63 (p<.003) and 1.19 (p<.001) standard deviation units, respectively, indicating better discriminant validity for CAT.Conclusions-Instead of using small fixed-length tests, clinicians can create item banks with a large item pool, and a small set of the items most relevant for a given individual can be administered with no loss of information, yielding a dramatic reduction in administration time and patient and clinician burden.Psychiatric measurement has been based primarily on subjective judgment and classical test theory. Typically, impairment level is determined by a total score, which requires that the same items be administered to all respondents. An alternative to administration of a full scale is disclosures The authors report no competing interests. This form of testing has recently emerged in mental health research (3,4). Procedures based on item response theory (5) can be used to obtain estimates for items (for example, difficulty or discrimination) and individuals (for example, severity of depression) to more efficiently identify suitable item subsets for each individual. This approach to testing is referred to as computerized adaptive testing (CAT) and is immediately applicable to psychiatric services (6-10). For example, a depression inventory can be administered adaptively, such that an individual responds only to items that are most appropriate for assessing his or her level of depression. The net result is that a small, optimal number of items is administered to the individual without loss of measurement precision. NIH Public AccessA complication of applying item response theory to psychiatric measurement problems is that unlike traditional ability testing (for example, mathematics achievement), for which approximately unidimensional scales are used, psychiatric...

show abstract

“…However, recent findings suggest that CAT could be applied to the MEOCS. Dodd, De Ayala, and Koch (1995) found the size of item bank needed is much lower when items are polytomously scored, as is done with the MEOCS. CAT has been done with several personality measures, including the Multidimensional Personality Questionnaire (Waller & Reise, 1989), the Minnesota Multiphasic Personality Inventory (Handel, Ben-Porath, & Watt, 1999), and with performance evaluation (Borman, Buck, Hanson, Motowidlo Stark, & Drasgow, 2001).…”

Section: The Advantages Oflrtto the Meocsmentioning

confidence: 95%

Comparing Two Versions of the MEOCS Using Differential Item Functioning

Truhon¹

2003

View full text Add to dashboard Cite

The public reporting burden for this collection of information is estimated to average 1 hour per response, including the time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and revievi/ing the collection of information. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing the burden, to Department of Defense, Washington Headquarters Services, Directorate for Information Operations and Reports (0704-0188), 1215 Jefferson Davis Highway, Suite 1204, Arlington, VA 22202-4302. Respondents should be aware that notwithstanding any other provision of law, no person shall be subject to any penalty for failing to comply with a collection of information if it does not display a currently valid 0MB control number. PLEASE DO NOT RETURN YOUR FORM TO THE ABOVE ADDRESS. REPORT DATE (DD-MM-YYYY) SPONSORING/MONITORING AGENCY NAME(S) AND ADDRESS(ES)Office of Naval Research 800 N. Quincy Street Arlington, VA 22032 SPONSOR/MONITOR'S ACRONYM(S) ONR SPONSOR/MONITOR'S REPORT NUMBER{S)N/A DISTRIBUTION/AVAILABILITY STATEMENTApproved for public release; distribution unlimited. SUPPLEMENTARY NOTES ABSTRACTAs part of a project to update the Military Equal Opportunity Climate Survey (MEOCS), items from eight of its scales (Sexual Harassment and Discrimination, Differential Command Behavior toward Minorities and Women, Positive Equal Opportunity (EO) Behavior, Racist/Sexist Behavior, Reverse Discrimination (Behavior), Discrimination against Minorities and Women, Reverse Discrimination (Attitude), and Attitudes toward Racial/Gender Separatism) have been rewritten to make them more neutral (e.g., replacing terms such as "men" and "women" with "gender"). A comparison was made of the psychometric characteristics of &e original and revised versions of these items using differential item functioning (DIF) from item response theory (IRT). DIF was foimd for the majority of the 40 items examined, although in many cases the DIF indicated improvements in the revised items. Implications for these scales and for the use of IRT with the MEOCS are discussed. (Attitude), and Attitudes toward Racial/Gender Separatism) have been rev^^ritten to make them more neutral (e.g., replacing terms such as "men" and "women" with "gender"). A comparison was made of the psychometric characteristics of the original and revised versions of these items using differential item functioning (DIF) from item response theory (IRT). DEF was found for the majority of the 40 items examined, although in many cases the DDF indicated improvements in the revised items. Implications for these scales and for the use of IRT with the MEOCS are discussed. SUBJECT TERMS Summer 2002Opinions expressed in this report are those of the author and should not be construed to represent the official position of DEOMI, the military Services, or the Department of Defense. Item response theory (IRT) arose, in part, to meet some of the failings of clas...

show abstract

Computerized Adaptive Testing With Polytomous Items

Cited by 107 publications

References 45 publications

Simulated computerized adaptive tests for measuring functional status were efficient with good discriminant validity in patients with hip, knee, or foot/ankle impairments

Simulated computerized adaptive tests for measuring functional status were efficient with good discriminant validity in patients with hip, knee, or foot/ankle impairments

Using Computerized Adaptive Testing to Reduce the Burden of Mental Health Assessment

Comparing Two Versions of the MEOCS Using Differential Item Functioning

Contact Info

Product

Resources

About