Judgment research and neuropsychological assessment: A narrative review and meta-analyses.

Abstract

The authors review judgment research in the area of neuropsychological assessment. Topics include the reliability and validity of judgments, the appropriateness of confidence ratings, the value of training and experience, the cognitive processes of neuropsychologists, and the use of decision aids, including automated assessment programs and statistical prediction rules. Most of the research is on reliability and validity. The results indicate that neuropsychologists frequently make reliable and moderately valid judgments. However, for several tasks, experimenters have not studied the reliability and validity of judgments.

Interrater Reliability of Clinical Ratings and Neurocognitive Diagnoses in HIV

“…A notable limitation of the White et al (2002) study is that the raters were provided with very limited information regarding what constituted ''cognitive impairment.'' The absence of detailed, manualized guidelines potentially promotes increased reliance on subjective judgment in assigning diagnoses, which may subsequently weaken IRR (Garb & Schramke, 1996). Accordingly, one might hypothesize that providing raters with explicit, standardized guidelines for judging impaired performance and assigning diagnostic classifications would improve IRR, especially in a multicenter study examining a neuropsychologically and neuromedically complex patient population (i.e., an HIV-infected sample).…”

“…Heaton et al (1994) . To facilitate the external validity of these initially promising findings, multiple raters with differing levels of experience from diverse clinical and research settings would need to be included (Garb & Schramke, 1996).…”

Computers in clinical assessment: Historical developments, present status, and future challenges

“…Other individuals (e.g., Garb & Schramke, 1996) consider computerized neuropsychological assessment programs "promising" (and also noted that there remains ample room for improvement). Fray, Robbins, and Sahakian's (1996) review would seem to suggest reason for optimism.…”

Neuropsychologists’ ability to predict distorted symptom presentation

“…Older studies on clinical prediction of distorted symptom presentation Faust, Hart, Guilmette, & Arkes, 1988;Heaton et al, 1978) were criticized for being too artificial (Bigler, 1990;Garb & Schramke, 1996;Schmidt, 1989). Critics pointed out that clinicians do not solely rely on test data to determine the presence of cognitive impairment or psychopathology.…”

“…These older studies are, however, not without limitations (see for a critical review, Garb & Schramke, 1996). They were carried out in an era when distorted symptom presentation was underresearched, and clinicians had only few tools to screen for it (e.g., Merten et al, 2013).…”