Performance of three freely available methods for extracting white matter hyperintensities: FreeSurfer, UBO Detector, and BIANCA

Abstract

White matter hyperintensities (WMH) of presumed vascular origin are frequently found in MRIs of healthy older adults. WMH are also associated with aging and cognitive decline. Here, we compared and validated three algorithms for WMH extraction: FreeSurfer (T1w), UBO Detector (T1w + FLAIR), and FSL's Brain Intensity AbNormality Classification Algorithm (BIANCA; T1w + FLAIR) using a longitudinal dataset comprising MRI data of cognitively healthy older adults (baseline N = 231, age range 64-87 years). As reference we manually segmented WMH in T1w, three-dimensional (3D) FLAIR, and two-dimensional (2D) FLAIR images which were used to assess the segmentation accuracy of the different automated algorithms. Further, we assessed the relationships of WMH volumes provided by the algorithms with Fazekas scores and age. FreeSurfer underestimated the WMH volumes and scored worst in Dice Similarity Coefficient (DSC = 0.434) but its WMH volumes strongly correlated with the Fazekas scores (r s = 0.73). BIANCA accomplished the highest DSC (0.602) in 3D FLAIR images. However, the relations with the Fazekas scores were only moderate, especially in the 2D FLAIR images (r s = 0.41), and many outlier WMH volumes were detected when exploring withinperson trajectories (2D FLAIR: ~30%). UBO Detector performed similarly to BIANCA in DSC with both modalities and reached the best DSC in 2D FLAIR (0.531) without requiring a tailored training dataset. In addition, it achieved very high associations with the Fazekas scores (2D FLAIR: r s = 0.80). In summary, our results emphasize the importance of carefully contemplating the choice of the WMH segmentation algorithm and MR-modality.

Decoding the Best Automated Segmentation Tools for Vascular White Matter Hyperintensities in the Aging Brain: A Clinician’s Guide to Precision and Purpose

“…SAMSEG was the tool with the highest ICC, followed by BIANCA and both LST algorithms. In this sense, the ICC results previously reported for LST-LPA, LST-LGA, and BIANCA are very heterogeneous, finding results similar to ours in some cases but also different in others (Griffanti et al, 2016;Heinen et al, 2019;Hotz et al, 2021;Tran et al, 2022).…”

“…We combined the Fazekas volume and total WMV correlations because together they are a good indicator of vascular damage, since the strong ceiling and floor effect of the Fazekas scale is offset by the discriminatory capacity of the total WMH volume. The correlation analyses between automatic and manual segmentations have been previously replicated by other studies carried out with LST-LPA (Heinen et al, 2019;Tran et al, 2022), LST-LGA (Heinen et al, 2019;Schmidt et al, 2012;Tran et al, 2022) and BIANCA (Griffanti et al, 2016;Hotz et al, 2021;Ling et al, 2018). Regarding SAMSEG algorithm, the correlation analysis with manually segmented volume was not assessed by the original article of Cerri and collaborators, and to the best of our knowledge it has not been depicted in any study.…”

“…Regarding SAMSEG algorithm, the correlation analysis with manually segmented volume was not assessed by the original article of Cerri and collaborators, and to the best of our knowledge it has not been depicted in any study. Although the results in the literature are heterogeneous, most studies report a slight tendency for these tools to provide smaller lesions volume when compared to manual segmentation (Griffanti et al, 2016;Heinen et al, 2019;Hotz et al, 2021;Ling et al, 2018). As can be seen in Figure 3, LST-LPA was the tool with the closest correlation with manual segmentation, closely followed by LST-LGA and SAMSEG.…”

“…For LST-LPA, LST-LGA and SAMSEG, we evaluated their performance at different thresholds (minimum probability criterion to accept a given voxel as lesion), and in the case of BIANCA we tested the algorithm with different training sets (for more details see Supplementary materials 1 and 2). Additionally, as previous studies (Hotz et al, 2021) recommended FreeSurfer's automatic segmentation of WMH over the T1 image, we also include it we Supplementary material 4. Following, we describe each procedure in detail.…”

Higher‐resolution quantification of white matter hypointensities by large‐scale transfer learning from 2D images on the JPSC‐AD cohort

“…Additionally, (Wei, Poirion, et al, 2019 ; Wei, Tran, et al, 2019 ) compared the predictive power of both T1 and FLAIR lesions—respectively segmented with FreeSurfer 6 and SPM‐LST—and concluded a “general equivalence between these two,” based on a similar association with abnormal β‐amyloid and tau, although other researchers (Hotz et al, 2021 ) assessed that WMH lesions labeled by FreeSurfer v6.0.1 are no substitute for manual effort. Still, a T1 image provides potentially more spatial precision, which could prove useful for accurate longitudinal monitoring of early smaller lesions, particularly with tailored methods (Sudre et al, 2017 ).…”