An evaluation of computerized adaptive testing for general psychological distress: combining GHQ-12 and Affectometer-2 in an item bank for public mental health research

Abstract

BackgroundRecent developments in psychometric modeling and technology allow pooling well-validated items from existing instruments into larger item banks and their deployment through methods of computerized adaptive testing (CAT). Use of item response theory-based bifactor methods and integrative data analysis overcomes barriers in cross-instrument comparison. This paper presents the joint calibration of an item bank for researchers keen to investigate population variations in general psychological distress (GPD).MethodsMultidimensional item response theory was used on existing health survey data from the Scottish Health Education Population Survey (n = 766) to calibrate an item bank consisting of pooled items from the short common mental disorder screen (GHQ-12) and the Affectometer-2 (a measure of “general happiness”). Computer simulation was used to evaluate usefulness and efficacy of its adaptive administration.ResultsA bifactor model capturing variation across a continuum of population distress (while controlling for artefacts due to item wording) was supported. The numbers of items for different required reliabilities in adaptive administration demonstrated promising efficacy of the proposed item bank.ConclusionsPsychometric modeling of the common dimension captured by more than one instrument offers the potential of adaptive testing for GPD using individually sequenced combinations of existing survey items. The potential for linking other item sets with alternative candidate measures of positive mental health is discussed since an optimal item bank may require even more items than these.Electronic supplementary materialThe online version of this article (doi:10.1186/s12874-016-0158-7) contains supplementary material, which is available to authorized users.

How did the 2008-11 financial crisis affect work-related common mental distress? Evidence from 393 workplaces in Great Britain

“…First, the index offers a more generalizable summary of the continued experience of common mental distress in the workplace since it does depend less on individuals' propensities to experience specific symptoms or their specific interpretations of an individual adjective. This interpretation is in line with the common use of such indices in epidemiology and individual differences research and especially such adjective lists have been shown to be largely exchangeable indicators for common mental distress (e.g., Böhnke and Croudace (2016); Jones et al (2016); Stochl et al (2016); Warr (1990)).…”

“…In our case the survey assessed as specific component, the amount of job-related mental distress an employee experiences. While the questions cover fairly typical adjectives used in other instruments as well (Stochl et al, 2016), they cover only negative descriptors, which means that an assessment of positive mental health was not possible (Böhnke and Croudace, 2016).…”

Development of a Microsoft Excel tool for one-parameter Rasch model of continuous items: an application to a safety attitude survey

“…Many studies [39–43] have reported that item response theory (IRT)-based computer adaptive testing (CAT) has the advantages of both long-form and short-form questionnaires [44–46] in precision and efficiency. Many employees already own mobile phones.…”

What Technology Can and Cannot Do to Support Assessment of Non-cognitive Skills

“…This allows more sensitivity to student ability levels and reduces the influence of small mistakes and lucky guesses on the final estimated ability. While computerized adaptive testing is most often used to measure cognitive abilities, it can also improve the measurement of other constructs, like personality (Makransky et al, 2013) and mental health (Becker et al, 2008; Stochl et al, 2016). Because adaptivity is an important facet of non-cognitive skills, test design and administration organizations such as the National Center for Education Statistics recommend adaptive tests in collaborative problem solving and other future assessments (Fiore et al, 2017).…”