Tractography dissection variability: what happens when 42 groups dissect 14 white matter bundles on the same dataset?

Abstract

White matter bundle segmentation using diffusion MRI fiber tractography has become the method of choice to identify white matter fiber pathways in vivo in human brains. However, like other analyses of complex data, there is considerable variability in segmentation protocols and techniques. This can result in different reconstructions of the same intended white matter pathways, which directly affects tractography results, quantification, and interpretation. In this study, we aim to evaluate and quantify the variability that arises from different protocols for bundle segmentation. Through an open call to users of fiber tractography, including anatomists, clinicians, and algorithm developers, 42 independent teams were given processed sets of human wholebrain streamlines and asked to segment 14 white matter fascicles on six subjects. In total, we received 57 different bundle segmentation protocols, which enabled detailed volume-based and streamline-based analyses of agreement and disagreement among protocols for each fiber pathway. Results show that even when given the exact same sets of underlying streamlines, the variability across protocols for bundle segmentation is greater than all other sources of variability in the virtual dissection process, including variability within protocols and variability across subjects. In order to foster the use of tractography bundle dissection in routine clinical settings, and as a fundamental analytical tool, future endeavors must aim to resolve and reduce this heterogeneity. Although external validation is needed to verify the anatomical accuracy of bundle dissections, reducing heterogeneity is a step towards reproducible research and may be achieved through the use of standard nomenclature and definitions of white matter bundles and well-chosen constraints and decisions in the dissection process.

Evaluating the reliability of human brain white matter tractometry

“…Subject robustness remains high despite differences in the spatial extent of bundles. We replicated previous findings that the definition of major bundles can vary in terms of their spatial extent (quantified via wDSC) (13,35,41,47), depending on the software implementation or the ODF model used. As we show, low wDSC robustness often corresponds to low profile robustness, and vice versa ( Fig 6B,C, Fig 4A,B, and Fig 5A,B).…”

“…Previous work has called into question the the reliability of neuroimaging analysis (e.g., (24,41,42)). We assessed the reliability of a specific approach, tractometry, which is grounded in decades of anatomical knowledge, and we demonstrate that this approach is reproducible, reliable and robust.…”

“…There are still significant challenges to robustness that arise from the way in which the major bundles are defined. This problem was highlighted in recent work that demonstrated that different researchers use different criteria to define bundles of streamlines that represent the same tract (41). In our case, this challenge is represented by the relatively low robustness between the waypoint ROI algorithm for bundle definition and the RecoBundles algorithm.…”

“…As highlighted in the recent work by Botvinik-Nezer et al (24) and in parallel by Schilling et al (41), inferences from even a single dataset can vary significantly, depending on the decisions and analysis pipelines that are used.The analysis approaches used in tractometry embody many assumptions made at the different stages of analysis: the model of the signal in each individual voxel, the manner in which streamlines are generated in tractography, the definition of bundles, and the extraction of tract profiles. While TRR is important, it does not guard against systematic errors in the analysis approach.…”

A taxonomy of the brain’s white matter: Twenty-one major tracts for the twenty-first century

“…The review is meant to provide a primer, a lookup table for students, educators and researchers interested in white matter by providing an introduction without dwelling on the technical aspects of any of the specific subfields that are involved in continuously contributing knowledge to understanding white matter. It is our hope that we are able to facilitate the expedited formation of a consensus white matter taxonomy (Schilling et al, 2020a). We suggest, as others have, that the formation of such a consensus is an important and necessary precursor to the widespread adoption of investigations of white matter as a standard component of neurobiological accounts of behavior, brain disorders, and changes in cognition across the lifespan.…”