Quantification of accuracy and precision of multi-center DTI measurements: A diffusion phantom and human brain study

Abstract

The inter-site and intra-site variability of system performance of MRI scanners (due to site-dependent and time-variant variations) can have significant adverse effects on the integration of multi-center DTI data. Measurement errors in accuracy and precision of each acquisition determine both the inter-site and intra-site variability. In this study, multiple scans of an identical isotropic diffusion phantom and of the brain of a travelling human volunteer were acquired at MRI scanners from the same vendor and with similar configurations at three sites. We assessed the feasibility of multi-center DTI studies by direct quantification of accuracy and precision of each dataset. Accuracy was quantified via comparison to carefully constructed gold standard datasets while precision (the within-scan variability) was estimated by wild bootstrap analysis. The results from both the phantom and human data suggest that the inter-site variation in system performance, although relatively small among scanners of the same vendor, significantly affects DTI measurement accuracy and precision and therefore the effectiveness for the integration of multi-center DTI measurements. Our results also highlight the value of a DTI-specific phantom in identifying and quantifying measurement errors due to site-dependent variations in the system performance, and its usefulness for quality assurance/quality control in multi-center DTI studies. In addition, we observed that the within-scan variability of each data acquisition, as assessed by wild bootstrap analysis, is of the same magnitude as the inter-site and intra-site variability. We propose that by weighing datasets based on their variability, as evaluated by wild bootstrap analysis, one can improve the quality of the dataset. This approach will provide a more effective integration of datasets from multi-center DTI studies.

Measuring the progression of upper motor neuron disease using diffusion MRI and its correlation with Clinical phenotype

“…We did not perform serial MRI in controls, so we cannot comment on whether the changes that we see on repeated testing differ from those of controls. Previous studies have reported that although inter-site DTI measurement variability was found to be quite high (Zhu, Hu et al 2011), the intra-subject variation was less than 1% in different brain regions using DTI (Magnotta, Matsui et al 2012). Our patients showed variation of more than 1%, therefore changes we observed are unlikely to be caused by measurement error alone.…”

Assessment of diffusion tensor image quality across sites and vendors using the American College of Radiology head phantom

“…With many centers contributing relatively few patients, variability between sites must be minimized. There have been several reports comparing diffusion measurements in multicenter studies 5 , 6 , 7 , 8 documenting significant variability for scanners of the same model using essentially identical technique and parameters. Many aspects of scanner performance have been reported to affect the measurement results and introduce discrepancies in the measurement (14) .…”

“…Repeatability and reproducibility data were not collected from sites participating in this study. Therefore, it is difficult to assess whether the variability of all the metrics is due to the systematic difference of machines from different vendors, as mentioned in the studies by Volimar et al, Zhu et al, Fox et al, and Teipel et al, 5 , 6 , 7 , 8 or due to random errors in the MRI measurements. Since the goal of this study was to evaluate a QC methodology for multicenter trial, knowing under what circumstance the site needs to undergo hardware/software adjustment is important.…”

“…These technique‐related effects may both mask and mimic biological changes in white matter. There have been several reports comparing diffusion measurements in multicenter studies 5 , 6 , 7 , 8 documenting significant variability for scanners of the same model using essentially identical technique and parameters. Conducting a multicenter study using quantitative analysis of diffusion tensor data requires a uniform quality control procedure to ensure that: (i) data from different sites are not biased and can be combined for statistical analysis; and (ii) data of substandard quality are excluded.…”

A quality assurance protocol for diffusion tensor imaging using the head phantom from American College of Radiology

“…These deficiencies may be overcome by including periodic scanning of human volunteers as part of a quality assurance procedure. 33 …”