Using Item and Test Information to Optimize Targeted Assessments of Psychological Distress

Pickett

et al. 2016

BMC Med Res Methodol

Self Cite

BackgroundRecent developments in psychometric modeling and technology allow pooling well-validated items from existing instruments into larger item banks and their deployment through methods of computerized adaptive testing (CAT). Use of item response theory-based bifactor methods and integrative data analysis overcomes barriers in cross-instrument comparison. This paper presents the joint calibration of an item bank for researchers keen to investigate population variations in general psychological distress (GPD).MethodsMultidimensional item response theory was used on existing health survey data from the Scottish Health Education Population Survey (n = 766) to calibrate an item bank consisting of pooled items from the short common mental disorder screen (GHQ-12) and the Affectometer-2 (a measure of “general happiness”). Computer simulation was used to evaluate usefulness and efficacy of its adaptive administration.ResultsA bifactor model capturing variation across a continuum of population distress (while controlling for artefacts due to item wording) was supported. The numbers of items for different required reliabilities in adaptive administration demonstrated promising efficacy of the proposed item bank.ConclusionsPsychometric modeling of the common dimension captured by more than one instrument offers the potential of adaptive testing for GPD using individually sequenced combinations of existing survey items. The potential for linking other item sets with alternative candidate measures of positive mental health is discussed since an optimal item bank may require even more items than these.Electronic supplementary materialThe online version of this article (doi:10.1186/s12874-016-0158-7) contains supplementary material, which is available to authorized users.

Section: Discussionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

An evaluation of computerized adaptive testing for general psychological distress: combining GHQ-12 and Affectometer-2 in an item bank for public mental health research

Štochl

Pickett

et al. 2016

BMC Med Res Methodol

Self Cite

“…This is in line with other efforts to condense the measurement of depression into one or two items for the purpose of efficient patient screening and monitoring [ 31 – 34 ]. The choice between long and short form assessment tools will depend on the context and purpose of the evaluation, balancing ease of data collection with the need for robust clinical diagnoses [ 35 ]. It remains unknown whether a single SMS depression score, as used in the ACUDep trial, can be considered a valid measure of depression and could consequently be recommended for use in research and evaluation in clinical practice.…”

Section: Introductionmentioning

confidence: 99%

Validity of single item responses to short message service texts to monitor depression: an mHealth sub-study of the UK ACUDep trial

Keding

Croudace

et al. 2015

BMC Med Res Methodol

Self Cite

BackgroundAn increasing number of research designs are using text messaging (SMS) as a means of self-reported symptom and outcome monitoring in a variety of long-term health conditions, including severity ratings of depressed mood. The validity of such a single item SMS score to measure latent depression is not currently known and is vital if SMS data are to inform clinical evaluation in the future.MethodsA sub-set of depressed participants in the UK ACUDep trial submitted a single SMS text score (R-SMS-DS) between 1 and 9 on how depressed they felt around the same time as completing the PHQ-9 depression questionnaire on paper at 3 months follow-up of the trial. Exploratory categorical data factor analysis (EFA) was used to ascertain the alignment of R-SMS-DS scores with the factor structure of the PHQ-9. Any response bias with regard to age or gender was assessed by differential item functioning (DIF) analysis.ResultsDepression scores based on the PHQ-9 and R-SMS-DS at 3 months were available for 337 participants (74 % female; mean age: 42 years, SD = 11.1), 213 of which completed the two outcomes within 6 days of each other. R-SMS-DS scores aligned with the underlying latent depression of the PHQ-9 (factor loading of 0.656) and in particular its affective rather than somatic dimension. The R-SMS-DS score was most strongly correlated with depressed mood (r = 0.607), feeling bad about oneself (r = 0.588) and anhedonia (r = 0.573). R-SMS-DS responses were invariant with respect to gender (p = 0.302). However, there was some evidence for age related response bias (p = 0.031), with older participants being more likely to endorse lower R-SMS-DS scores than younger ones.ConclusionsThe R-SMS-DS used in the ACUDep trial was found to be a valid measure of latent affective depression with no gender related response bias. This text message item may therefore represent a useful assessment and monitoring tool meriting evaluation in further research. For future study designs we recommend the collection of outcome data by new health technologies in combination with gold standard instruments to ensure concurrent validity.

“…For such applications, a highly reliable test is needed to allow for clear decisions about whether an individual is above or below a relevant severity threshold. For this, the confidence interval around the individual severity level or the relevant threshold needs to be small: this decreases the number of cases for which the severity threshold is included in the confidence interval around the individual’s severity level (or the severity level lies within the interval around the threshold, respectively) [ 3 , 38 ]. A reliability of 0.84 seems rather low for such decisions.…”

Section: Discussionmentioning

confidence: 99%

“…In HALS, due to the two-waves of GHQ responses, we report this for both baseline and follow-up (in Table 2 ). The results indicated that, to achieve a high level of reliability 2 [ 36 – 38 ] for a latent construct score (>0.9), almost all GHQ items need to be administered. This result held regardless of the method of θ estimation or item selection algorithm chosen.…”

Section: Methodsmentioning

confidence: 99%

Computerized adaptive testing of population psychological distress: simulation-based evaluation of GHQ-30

Štochl

Soc Psychiatry Psychiatr Epidemiol

Pickett

et al. 2015

Self Cite

PurposeGoldberg’s General Health Questionnaire (GHQ) items are frequently used to assess psychological distress but no study to date has investigated the GHQ-30’s potential for adaptive administration. In computerized adaptive testing (CAT) items are matched optimally to the targeted distress level of respondents instead of relying on fixed-length versions of instruments. We therefore calibrate GHQ-30 items and report a simulation study exploring the potential of this instrument for adaptive administration in a longitudinal setting.MethodsGHQ-30 responses of 3445 participants with 2 completed assessments (baseline, 7-year follow-up) in the UK Health and Lifestyle Survey were calibrated using item response theory. Our simulation study evaluated the efficiency of CAT administration of the items, cross-sectionally and longitudinally, with different estimators, item selection methods, and measurement precision criteria.ResultsTo yield accurate distress measurements (marginal reliability at least 0.90) nearly all GHQ-30 items need to be administered to most survey respondents in general population samples. When lower accuracy is permissible (marginal reliability of 0.80), adaptive administration saves approximately 2/3 of the items. For longitudinal applications, change scores based on the complete set of GHQ-30 items correlate highly with change scores from adaptive administrations.ConclusionsThe rationale for CAT-GHQ-30 is only supported when the required marginal reliability is lower than 0.9, which is most likely to be the case in cross-sectional and longitudinal studies assessing mean changes in populations. Precise measurement of psychological distress at the individual level can be achieved, but requires the deployment of all 30 items.