Paper is not enough: Crowdsourcing the T1 mapping common ground via the ISMRM reproducibility challenge

Abstract

We present the results of the ISMRM 2020 joint Reproducible Research and Quantitative MR study groups reproducibility challenge on T1 mapping in phantom and human brain. T1 mapping, a widely used quantitative MRI technique, exhibits inconsistent tissue-specific values across protocols, sites, and vendors. The challenge aimed to assess the reproducibility of a well-established inversion recovery T1 mapping technique, with acquisition details published solely as a PDF, on a standardized phantom and in human brains. Participants acquired T1 mapping data from MRIs of three manufacturers at 3T, resulting in 39 phantom datasets and 56 datasets from healthy human subjects. The T1 inter-submission variability was twice as high as the intra-submission variability in both phantoms and human brains, indicating that the acquisition details in the selected paper were insufficient to reproduce a quantitative MRI protocol. This study reports the inherent uncertainty in T1 measures across independent Boudreau et al. ( 2024). Paper is not enough: Crowdsourcing the T1 mapping common ground via the ISMRM reproducibility challenge. NeuroLibre Reproducible Preprints, 23. https://doi.org/10.55458/neurolibre.00023. Additionally, Kathryn Keenan, Zydrunas Gimbutas, and Andrew Dienstfrey from NIST provided their code to generate the ROI template for the ISMRM/NIST phantom. Dylan Roskams-Edris and Gabriel Pelletier from the Tanenbaum Open Science Institute (TOSI) offered valuable insights and guidance related to data ethics and data sharing in the context of this international multi-center conference challenge. The 2020 RRSG study group committee members who launched the challenge,