Assessing intrarater, interrater and test–retest reliability of continuous measurements

Abstract

In this paper we review the problem of defining and estimating intrarater, interrater and test-retest reliability of continuous measurements. We argue that the usual notion of product-moment correlation is well adapted in a test-retest situation, whereas the concept of intraclass correlation should be used for intrarater and interrater reliability. The key difference between these two approaches is the treatment of systematic error, which is often due to a learning effect for test-retest data. We also consider the reliability of a sum and a difference of variables and illustrate the effects on components. Further, we compare these approaches of reliability with the concept of limits of agreement proposed by Bland and Altman (for evaluating the agreement between two methods of clinical measurements) and show how product-moment correlation is related to it. We then propose new kinds of limits of agreement which are related to intraclass correlation. A test battery to study the development of neuro-motor functions in children and adolescents illustrates our purpose throughout the paper.

Regional activation of the human medial temporal lobe during intentional encoding of objects and positions

“…Thus, ROIs in a total of 26 structural scans were drawn (by TZR) for intrarater comparison. An intraclass correlation test (Rousson et al, 2002) on each ROI volume showed a mean r = 0.884 for all regions (range 0.615 to 0.916). The lowest values were for the bilateral temporopolar cortex, all other correlations exceeded 0.8.…”

Intersession reliability of a posturo-stabilometric test, using a force platform

“…whether the resulting measurements of this test are reproducible among sessions. As it was reasonable to believe that no systematic error occurred, the intraclass correlation coefficient (ICC) was estimated to ensure test-retest reliability between sessions (Rousson et al, 2002) ICC may be defined as the percentage of variability in measurements explained by the subject effect thus the larger this percentage the better the reliability. An ICC larger than 0.75 indicates good to excellent reliability, an ICC included between 0.40 and 0.75 indicates fair to good reliability, and finally an ICC lower than 0.40 indicates poor reliability (Corriveau et al, 2000).…”

Healthy aging attenuates task-related specialization in the human medial temporal lobe

“…Thus, ROIs in a total of 26 structural scans were drawn for intrarater comparison. An intraclass correlation test (Rousson et al, 2002) on each ROI volume showed a mean r ϭ 0.884 for all regions (range 0.615-0.916). The lowest values were for the bilateral temporopolar cortex, all other correlations exceeded 0.8.…”