Test–retest: Agreement or reliability?

Abstract

Test-retest is a concept that is routinely evaluated during the validation phase of many measurement tools. However, this term covers at least two related but very different concepts: reliability and agreement. Reliability is the ability of a measure applied twice upon the same respondents to produce the same ranking on both occasions. Agreement requires the measurement tool to produce twice the same exact values. An analysis of research papers showed that the distinction between both concepts remains anything but clear, and that the current practice is to evaluate reliability only, generally on the basis of the sole Pearson's correlation. This practice is very problematic in the context of longitudinal studies because it becomes difficult to determine whether a difference between two successive observations is attributable to a real change of the respondents or only to the characteristics of the measurement tool, which then leads to a possible misinterpretation of the results. More focus should be given on the real interpretation of linear correlation, and when agreement is required in addition to reliability, then correct alternative, such as the Bland-Altman plot, should be more generally used.

Retracted: Test‐Retest Reliability of Dynamic Balance Performance‐Based Measures Among Adults With a Unilateral Lower‐Limb Amputation

“…Although the interpretation of ICC values can be controversial, the ICC point estimates in this study ranged from 0.69-0.97, which is generally considered good-to-excellent [52,53]. Some concerns with prior results presentation include the non-utilization of the Equation (3,1 or 3,k; depending on whether the score was based on a single trial [1,3] or several trials [3,k]), because it is most closely tied to the error mean square calculation of the SEM [18]; failure to present confidence intervals; using the Spearman correlation coefficient, which has been discouraged because of the lack of detection of systematic error; and non-calculation and description of the SEM and the MDC [18,54]. In this study, we sought to follow reporting guidelines and thus provide outcome measurement data that may be used in daily clinical practice to evaluate patient status, assess decline or progress, and assist with clinical decision-making [18,54].…”

Topographic mapping of pain sensitivity of the lower back – a comparison of healthy controls and patients with chronic non-specific low back pain

“…Test/re-test reliability was assessed using Intra-class Correlation Coefficients ICC3.2 for single fixed raters (similar to Pearsons R) as a measure of correlation and Concordance Correlation Coefficient as a measure of agreement, as recommended by Carrasco [49]. Graphical illustrations of test/re-test reliability are provided as customized Bland-Altman plots, as recommended by Berchtold [50].…”

Test–retest reliability measures for curve data: an overview with recommendations and supplementary code

“…This search was directed to literature related to the analysis of biomechanics and to statistics in general. Since an ambiguity in terminology exists in the literature where alternative terms like repeatability, consistency, agreement, concordance, reproducibility, and stability are used in preference to reliability (Atkinson & Nevill, 1998;Berchtold, 2016;De Vet et al, 2006), these terms were also considered in the searches. The reference lists within the relevant articles were also reviewed.…”