Statistical evaluation of test-retest studies in PET brain imaging

Abstract

BackgroundPositron emission tomography (PET) is a molecular imaging technology that enables in vivo quantification of metabolic activity or receptor density, among other applications. Examples of applications of PET imaging in neuroscience include studies of neuroreceptor/neurotransmitter levels in neuropsychiatric diseases (e.g., measuring receptor expression in schizophrenia) and of misfolded protein levels in neurodegenerative diseases (e.g., beta amyloid and tau deposits in Alzheimer’s disease). Assessment of a PET tracer’s test-retest properties is an important component of tracer validation, and it is usually carried out using data from a small number of subjects.ResultsHere, we investigate advantages and limitations of test-retest metrics that are commonly used for PET brain imaging, including percent test-retest difference and intraclass correlation coefficient (ICC). In addition, we show how random effects analysis of variance, which forms the basis for ICC, can be used to derive additional test-retest metrics, which are generally not reported in the PET brain imaging test-retest literature, such as within-subject coefficient of variation and repeatability coefficient. We reevaluate data from five published clinical PET imaging test-retest studies to illustrate the relative merits and utility of the various test-retest metrics. We provide recommendations on evaluation of test-retest in brain PET imaging and show how the random effects ANOVA based metrics can be used to supplement the commonly used metrics such as percent test-retest.ConclusionsRandom effects ANOVA is a useful model for PET brain imaging test-retest studies. The metrics that ensue from this model are recommended to be reported along with the percent test-retest metric as they capture various sources of variability in the PET test-retest experiments in a succinct way.

Kinfitr – an open source tool for reproducible PET modelling: validation and evaluation of test-retest reliability

“…We found excellent agreement between binding estimates computed using both tools, with a median ICC of 0.98 (range: 0.81-1.00) ( Table 1, Supplementary Materials S3) [50]. Likewise, we found high correlations between kinfitr and PMOD, with a median correlation coefficient of 0.99 (range: 0.95-1.00) ( Table 1).…”

“…This metric can be considered as an approximation of the WSCV above. While not as useful as the WSCV [50], AV has traditionally been applied within the PET field, and is included for historical comparability.…”

We need to talk about reliability: Making better use of test-retest studies for study design and interpretation

“…Each measurement is made with an associated error, which can be described by its standard error ( σ e ). It can be estimated as the square root of the within subject mean sum of squares (MS W ), which is used in the calculation of the ICC above (Baumgartner et al 2018). where n represents the number of participants, i represents the subject number, j represents the measurement number, k represents the number of measurements per subject, y represents the outcome and ȳ i represents the mean outcome for that subject.…”

“…The relative or absolute uncertainty can be used to calculate the smallest detectable difference (SDD) between two measurements in a given subject which could be considered sufficiently large that it is unlikely to have been due to chance alone (say, according to a 95% confidence interval, i.e. using z (1− α /2) =1.96 below) (Weir 2005; Baumgartner et al 2018). This is calculated using the following equation.…”

“…Each measurement is made with an associated error, which can be described by its standard error ( σ e ). It can be estimated as the square root of the within subject mean sum of squares (MS W ), which is used in the calculation of the ICC above (Baumgartner et al 2018).…”