Computerized adaptive measurement of depression: A simulation study

Gardner, William; Shear, Katherine; Kelleher, Kelly J.; Pajer, Kathleen; Mammen, Oommen; Buysse, Daniel J.; Frank, Ellen

doi:10.1186/1471-244x-4-13

Cited by 66 publications

(67 citation statements)

References 46 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Similar results were obtained for live CAT testing. It should be noted that similar results were obtained with a CAT version of the Beck Depression Inventory (57) and with other measures (3,4).…”

Section: Discussionsupporting

confidence: 58%

“…Procedures based on item response theory (5) can be used to obtain estimates for items (for example, difficulty or discrimination) and individuals (for example, severity of depression) to more efficiently identify suitable item subsets for each individual. This approach to testing is referred to as computerized adaptive testing (CAT) and is immediately applicable to psychiatric services (6)(7)(8)(9)(10). For example, a depression inventory can be administered adaptively, such that an individual responds only to items that are most appropriate for assessing his or her level of depression.…”

mentioning

confidence: 99%

See 1 more Smart Citation

Using Computerized Adaptive Testing to Reduce the Burden of Mental Health Assessment

Gibbons

Weiss

Kupfer

et al. 2008

Psychiatric Services

119

View full text Add to dashboard Cite

Objective-This study investigated the combination of item response theory and computerized adaptive testing (CAT) for psychiatric measurement as a means of reducing the burden of research and clinical assessments.Methods-Data were from 800 participants in outpatient treatment for a mood or anxiety disorder; they completed 616 items of the 626-item Mood and Anxiety Spectrum Scales (MASS) at two times. The first administration was used to design and evaluate a CAT version of the MASS by using post hoc simulation. The second confirmed the functioning of CAT in live testing.Results-Tests of competing models based on item response theory supported the scale's bifactor structure, consisting of a primary dimension and four group factors (mood, panic-agoraphobia, obsessive-compulsive, and social phobia). Both simulated and live CAT showed a 95% average reduction (585 items) in items administered (24 and 30 items, respectively) compared with administration of the full MASS. The correlation between scores on the full MASS and the CAT version was .93. For the mood disorder subscale, differences in scores between two groups of depressed patients-one with bipolar disorder and one without-on the full scale and on the CAT showed effect sizes of .63 (p<.003) and 1.19 (p<.001) standard deviation units, respectively, indicating better discriminant validity for CAT.Conclusions-Instead of using small fixed-length tests, clinicians can create item banks with a large item pool, and a small set of the items most relevant for a given individual can be administered with no loss of information, yielding a dramatic reduction in administration time and patient and clinician burden.Psychiatric measurement has been based primarily on subjective judgment and classical test theory. Typically, impairment level is determined by a total score, which requires that the same items be administered to all respondents. An alternative to administration of a full scale is disclosures The authors report no competing interests. This form of testing has recently emerged in mental health research (3,4). Procedures based on item response theory (5) can be used to obtain estimates for items (for example, difficulty or discrimination) and individuals (for example, severity of depression) to more efficiently identify suitable item subsets for each individual. This approach to testing is referred to as computerized adaptive testing (CAT) and is immediately applicable to psychiatric services (6-10). For example, a depression inventory can be administered adaptively, such that an individual responds only to items that are most appropriate for assessing his or her level of depression. The net result is that a small, optimal number of items is administered to the individual without loss of measurement precision. NIH Public AccessA complication of applying item response theory to psychiatric measurement problems is that unlike traditional ability testing (for example, mathematics achievement), for which approximately unidimensional scales are used, psychiatric...

show abstract

Section: Discussionsupporting

confidence: 58%

mentioning

confidence: 99%

Using Computerized Adaptive Testing to Reduce the Burden of Mental Health Assessment

Gibbons

Weiss

Kupfer

et al. 2008

Psychiatric Services

119

View full text Add to dashboard Cite

show abstract

“…This particular advantage may be especially relevant in studies where developmental differences in the phenomenology of depression are the focus of interest (see Weiss & Garber, 2003 for a review of such studies). Lastly, using IRT technologies, there is potential for a clinical scale to be substantially shortened through the adoption of an adaptive testing approach (Gardner et al, 2004). With adaptive testing (computerised adaptive testing-CAT) a software program uses IRT parameters to select items in a sequence that optimally gain information on the severity score for each respondent, by tailoring questions to the likely level of the individual's score.…”

Section: Discussionmentioning

confidence: 99%

The Short Mood and Feelings Questionnaire (SMFQ): A Unidimensional Item Response Theory and Categorical Data Factor Analysis of Self-Report Ratings from a Community Sample of 7-through 11-Year-Old Children

Sharp

Goodyer

Croudace

2006

J Abnorm Child Psychol

293

199

View full text Add to dashboard Cite

Item response theory (IRT) and categorical data factor analysis (CDFA) are complementary methods for the analysis of the psychometric properties of psychiatric measures that purport to measure latent constructs. These methods have been applied to relatively few child and adolescent measures. We provide the first combined IRT and CDFA analysis of a clinical measure (the Short Mood and Feelings Questionnaire-SMFQ) in a community sample of 7-through 11-year-old children. Both latent variable models supported the internal construct validity of a single underlying continuum of severity of depressive symptoms. SMFQ items discriminated well at the more severe end of the depressive latent trait. Item performance was not affected by age, although age correlated significantly with latent SMFQ scores suggesting that symptom severity increased within the age period of 7-11. These results extend existing psychometric studies of the SMFQ and confirm its scaling properties as a potential dimensional measure of symptom severity of childhood depression in community samples. KEY WORDS:Screening; childhood depression; SMFQ; item response theory; categorical data factor analysis.Over the last 40 years, the methods used to evaluate the psychometric basis of ability tests, health care surveys, and multi-item screening instruments has changed dramatically. Whilst the methodology of classical test theory (CTT) has served test development well, item response/latent trait theory (IRT) approaches have become more mainstream as the technical basis for measurement theory, test construction and scale evaluation (Embretson & Reise, 2000). Although moves towards adoption of more appropriate, non-linear and categorical data factor analysis (CDFA) models have been most apparent in ed- Costello, 2002;Cooke & Michie, 1997;Lambert et al., 2003;Patton, Carlin, Shao, Hibbert, & Bowes, 1997;Santor, Ramsay, & Zuroff, 1994). Currently there are very few reports that have applied such methodologies in samples of young children (Cheong & Raudenbush, 2000). One reason for the under-exploitation of such methodologies may be because researchers have not been introduced to the potential and practicalities of these methods (Rouse, Finger, & Butcher, 1999) and are therefore unaware of the advantages they offer over conventional (CTT) methods (van der Linden & Hambleton, 1997). Although CTT is often included in the curriculum of both clinical and applied psychologists, IRT is rarely taught, and has had less coverage in mainstream 380Sharp, Goodyer, and Croudace psychology journals (Embretson & Reise, 2000). We provide an application of IRT and categorical data factor analysis (CDFA) methods to a commonly used self-report measure of depressive symptoms in children, the Short Mood and Feelings Questionnaire (SMFQ; Angold et al., 1995). As such, the aim of this paper was to scrutinize the internal construct validity of the SMFQ. To this end, we used latent variable models implemented with both an IRT and appropriate factor analysis framework (CDFA). Within a lat...

show abstract

“…Older individuals who are frailer may also tire more easily, making the use of more time-consuming procedures impractical. Computerized presentation has the potential to reduce respondent burden (Gardner et al, 2004), particularly if the system uses an adaptive approach to reduce the number of items (Fliege et al, 2005) but at present there are no such measures available for mood assessment, only depression (Gardner et al, 2004). Impairments of memory and other cognitive functions can also limit compliance with, or understanding of, relatively complex procedures involved in mood assessment in older adults.…”

Section: The Challenges Of Mood Assessmentmentioning

confidence: 99%