The effect of training on rater reliability on the scoring of the NART

Abstract

Training in administration of the NART improves raters' accuracy and use of the pronunciation guide. This offers an alternative to the suggestion of improving the NART's reliability by replacing lower reliability words and therefore would avoid the need to re-standardize a modified test.

How robust is performance on the National Adult Reading Test following traumatic brain injury?

“…The performance of all the participants was re-scored by a person who was unaware of the purpose of the study. This person was not a health professional, and was trained in the administration of the NART using the methods suggested by Alcott, Swann, and Grafham (1999).…”

Stability of estimated IQ across mood state in patients with bipolar disorder

“…The number of incorrectly pronounced words was then tabulated and entered into the standardized scoring equation to calculate estimated IQ (1). Because test reliability has been demonstrated to increase when raters are trained on this task (24), all raters had previously been trained and high inter‐rater reliability had been achieved ( k > 0.9). Follow‐up raters were blind to the ANART‐derived IQ scores previously obtained during the patient's initial assessment.…”

Assessing pain in older adults with dementia

“…When inferring pain in others, providers should be trained to execute the pain assessment or screening by the preferred method of using a pain assessment tool as a guide. Adequate training on pain assessment improves the provider's accuracy and familiarity with the tool during assessments (Alcott et al, 1999;Cusick et al, 2005;Roch et al, 2012). Additionally, trained providers will typically have more familiarity with an assessment tool, as training provides more opportunities for practice, feedback, and role modeling (Cusick et al, 2005).…”

“…Additionally, trained providers will typically have more familiarity with an assessment tool, as training provides more opportunities for practice, feedback, and role modeling (Cusick et al, 2005). Training has also been shown to improve the provider's overall performance and increase inter-rater reliability scores, particularly when being trained by an experienced trainer (Alcott et al, 1999;Mist et al, 2009). Alcott et al (1999) explored the benefits of training on word recognition, finding that training improved the rater's accuracy and pronunciation of low reliability words.…”

“…Training has also been shown to improve the provider's overall performance and increase inter-rater reliability scores, particularly when being trained by an experienced trainer (Alcott et al, 1999;Mist et al, 2009). Alcott et al (1999) explored the benefits of training on word recognition, finding that training improved the rater's accuracy and pronunciation of low reliability words. Similarly, Mist et al (2009) also identified an enhanced ability to accurately diagnose patients using Traditional Chinese Medicine following staff training.…”