Effects of Examiner Error on Neuropsychological Test Results in a Multi-site Study

Abstract

This study compared the difference between original and "corrected" neuropsychological test scores at baseline and following 1 year of experience in 17 non-psychology trained examiners. Test protocols were reviewed for errors in instruction, administration, recording, and scoring. Fewer than 3% of the test scaled scores showed a correction of greater than 1 SD. At baseline, individual test scores that changed T-score classification occurred on Digit Symbol, Trails B, and Logical Memory I and II. At one year, significant classification changes remained for Logical Memory I and II (25% and 15%). Scoring of subjective tests remains problematic and centralized re-scoring is recommended.

The Quality Assurance and Quality Control Protocol for Neuropsychological Data Collection and Curation in the Ontario Neurodegenerative Disease Research Initiative (ONDRI) Study

“…In addition, the standard administration and scoring instructions outlined in the test manual are simple and straightforward, and the protocol reduces the potential for variations in administration and errors (e.g., all items are administered, with no basal or discontinue rules; there are no acceptable alternative responses and the semantic cue always proceeds the phonemic cue, eliminating the need for clinical judgment; and time limits per item/cue are much shorter). As demonstrated here and elsewhere (Kozora et al, 2008), measures with a subjective component are more prone to error. Therefore, selecting tasks that minimize the need for "clinical judgment" is advantageous (e.g., the NAB naming over the BNT).…”

“…were corrected and approved immediately by the double-scorer and/or data entry verifier. On tasks with a subjective scoring component (i.e., Brief Visuospatial Memory Task-Revised [BVMT-R], Vocabulary, and Verbal Fluency), which are more susceptible to error (Kozora et al, 2008), the original score was overturned only after consensus was reached between the initial and double-scorer on reviewing the study manual and/or consulting with a member of the neuropsychology management team.…”

Improving the quality of cognitive screening assessments: ACEmobile, an iPad‐based version of the Addenbrooke's Cognitive Examination‐III

“…The psychometric properties of cognitive screening tools for dementia are routinely reported, yet far less is known about the clinician's ability to administer and score these tests accurately. Evidence suggests that users make many more errors than expected [1–3]. There is surprisingly little detail in the literature on how well the cognitive screening tests perform in the hands of the clinicians for whom they are designed.…”

Developing Proficiency in Standardized Cognitive Assessment Scoring: How Much Is Enough?

“…For example, cognitive assessments are used to make decisions about the eligibility for various remedial services for students (Gilmore & Campbell, 2009). Unfortunately, the complex scoring procedures involved when producing scores from examinee responses often result in a significant number of scoring errors on the record form (Allard & Faust, 2000; Alper, 2012; Kozora et al, 2007; Kozora, Kongs, Hampton, & Zhang, 2008; Simons, Goddard, & Patton, 2002). When psychologists make errors in their scoring, the consequence could be misdiagnosis or ineffective resource allocation.…”