Joint Consideration of Validity Indicators Embedded in Conners’ Adult ADHD Rating Scales (CAARS)

Abstract

A decade of research has both illustrated the need for accurate clinical assessment of adult ADHD and brought forward a series of validity indicators assisting this diagnostic process. Several of these indicators have been embedded into Conners’ Adult ADHD Rating Scales (CAARS). As their different theoretical underpinnings offer the opportunity of possible synergy effects, the present study sought to examine whether the item- or index-wise combination of multiple validity indicators benefits classification accuracy. A sample of controls (n = 856) and adults with ADHD (n = 72) answered the CAARS, including the ADHD Credibility Index (ACI) honestly, while a group of instructed simulators (n = 135) completed the instrument as though they had ADHD. First, original CAARS items, which are part of the CAARS Infrequency Index (CII), and items drawn from the ACI were combined into a new CII-ACI-Compound Index. Secondly, existing validity indicators, including suspect T-score elevations and the CII, were considered in combination. Both approaches were evaluated in terms of sensitivity and specificity. The combination of four CII and five ACI items into the CII-ACI-Compound Index yielded a sensitivity between 41 and 51% and an estimated specificity above 87%. Suspect T-score elevations on all three DSM scales emerged as another potentially useful validity indicator with a sensitivity of 45 to 46% and a specificity > 90%. Deeming examinees non-credible whenever two or more validity indicators showed suspect results ensured low false-positive rates (< 10%), but reduced sensitivity significantly. Classifying respondents as non-credible as soon as any given indicator fell into the suspect range resulted in frequent false positives (> 11% of misclassified adults with ADHD). Depending on whether high specificity or high sensitivity is prioritized, such combined considerations offer valuable additions to individual validity indicators. High sensitivity provided by “either/or” combinations could prove useful in screening settings, whereas high stakes settings could benefit from “and” combinations.

A review of behavioural rating scales for the assessment of Attention Deficit/Hyperactivity Disorder among adults in research

“…The CII-ACI-Compound Index had a high specificity of 87% to 92%, a moderate sensitivity of 41% to 50%, and PPV ranging from 15% to 87% and NPV ranging from 60% to 97%, varying between low-to-high and moderate-to-high and very high (Becke et al, 2022).…”

“…The authors also created the CII-ACI-Compound Index, a combination of items from both validity indexes. The CII-ACI-Compound Index had a high specificity of 87% to 92%, a moderate sensitivity of 41% to 50%, and PPV ranging from 15% to 87% and NPV ranging from 60% to 97%, varying between low-to-high and moderate-to-high and very high (Becke et al, 2022). Instead of utilising embedded validity indices to detect feigning of ADHD symptoms, Courrégé and colleagues (2019) developed a self-administered measure known as the ADHD Symptom Infrequency Scale (ASIS).…”

“…Although the ACI had very-high sensitivity, its specificity was found to be low-to-moderate. To improve its overall classification accuracy, the authors of the study developed the CII-ACI-Compound, which showed a very high sensitivity but only a moderate specificity (Becke et al, 2022). Although these indexes show promise, further research is necessary to improve their accuracy.…”