Correlation or Limits of Agreement? Applying the Bland-Altman Approach to the Comparison of Cognitive Screening Instruments

Abstract

Background/Aims: Calculation of correlation coefficients is often undertaken as a way of comparing different cognitive screening instruments (CSIs). However, test scores may correlate but not agree, and high correlation may mask lack of agreement between scores. The aim of this study was to use the methodology of Bland and Altman to calculate limits of agreement between the scores of selected CSIs and contrast the findings with Pearson's product moment correlation coefficients between the test scores of the same instruments. Methods: Datasets from three pragmatic diagnostic accuracy studies which examined the Mini-Mental State Examination (MMSE) vs. the Montreal Cognitive Assessment (MoCA), the MMSE vs. the Mini-Addenbrooke's Cognitive Examination (M-ACE), and the M-ACE vs. the MoCA were analysed to calculate correlation coefficients and limits of agreement between test scores. Results: Although test scores were highly correlated (all >0.8), calculated limits of agreement were broad (all >10 points), and in one case, MMSE vs. M-ACE, was >15 points. Conclusion: Correlation is not agreement. Highly correlated test scores may conceal broad limits of agreement, consistent with the different emphases of different tests with respect to the cognitive domains examined. Routine incorporation of limits of agreement into diagnostic accuracy studies which compare different tests merits consideration, to enable clinicians to judge whether or not their agreement is close.

Converting cognitive screening instrument test scores to MMSE scores: regression equations

“…Using the datasets of several pragmatic diagnostic test accuracy studies undertaken in a secondary care Cognitive Function Clinic (Abdel‐Aziz and Larner ; Larner , ), regression equations of the form y = a + bx were calculated, where y , the dependent or outcome variable, was approximate MMSE score; x , the independent or explanatory variable, was score on different CSIs with which MMSE was being compared, namely Montreal Cognitive Assessment (MoCA), Mini‐Addenbrooke's Cognitive Examination (MACE), Six‐item Cognitive Impairment Test (6CIT) and the informant AD8; and a is the intercept and b the slope or gradient (regression coefficient) of the regression equation. Pearson product moment correlation coefficients were also calculated (Table ).…”

Association between Montreal Cognitive Assessment Sub-Item Scores and Corresponding Cognitive Test Performance in Patients with Frontotemporal Dementia and Related Disorders

“…Some comparisons of different cognitive tests have used the Bland-Altman approach originally developed to assess agreement between tests measuring the same unit (i.e., mL/min). This approach was recently applied to assess agreement between the total scores on the MMSE, MoCA, and Addenbrooke's Cognitive Exam Revised (ACE-R) [24]. An earlier study applied the Bland-Altman approach to assess agreement between total scores on a longer cognitive test, the ADAS-Cog, with the MMSE and CDR using standardized scores given variability in ranges across the tests [25].…”

MACE for the Diagnosis of Dementia and MCI: 3-Year Pragmatic Diagnostic Test Accuracy Study

“…In a nephrology clinic cohort of patients with chronic kidney disease and diabetes (n = 112), the sensitivity and specificity for a dementia diagnosis were 0.96 and 0.84, respectively, at the cut point < 25/30 [6]. Studies in a dedicated cognitive disorders clinic have suggested superiority of the MACE to MMSE [7][8][9] and single-item cognitive screening questions [10] and equivalence to the Montreal Cognitive Assessment (MoCA) [9,11,12], and a better performance in those aged > 65 years, with a higher prevalence of cognitive impairment [13]. In a study of patients with mild AD (n = 92) and healthy controls (n = 68), MACE had an area under the receiver operating characteristic (AUC ROC) curve of 0.939, which was superior to ACE-III (0.897), Rowland Universal Dementia Assessment Scale (0.889), MMSE (0.874), Memory Impairment Screen (0.866), and MoCA (0.856) [14].…”