A novel image augmentation based on statistical shape and intensity models: application to the segmentation of hip bones from CT images

Abstract

Background The collection and annotation of medical images are hindered by data scarcity, privacy, and ethical reasons or limited resources, negatively affecting deep learning approaches. Data augmentation is often used to mitigate this problem, by generating synthetic images from training sets to improve the efficiency and generalization of deep learning models. Methods We propose the novel use of statistical shape and intensity models (SSIM) to generate augmented images with variety in both shape and intensity of imaged structures and surroundings. The SSIM uses segmentations from training images to create co-registered tetrahedral meshes of the structures and to efficiently encode image intensity in their interior with Bernstein polynomials. In the context of segmentation of hip joint (pathological) bones from retrospective computed tomography images of 232 patients, we compared the impact of SSIM-based and basic augmentations on the performance of a U-Net model. Results In a fivefold cross-validation, the SSIM augmentation improved segmentation robustness and accuracy. In particular, the combination of basic and SSIM augmentation outperformed trained models not using any augmentation, or relying exclusively on a simple form of augmentation, achieving Dice similarity coefficient and Hausdorff distance of 0.95 [0.93–0.96] and 6.16 [4.90–8.08] mm (median [25th–75th percentiles]), comparable to previous work on pathological hip segmentation. Conclusions We proposed a novel augmentation varying both the shape and appearance of structures in generated images. Tested on bone segmentation, our approach is generalizable to other structures or tasks such as classification, as long as SSIM can be built from training data. Relevance statement Our data augmentation approach produces realistic shape and appearance variations of structures in generated images, which supports the clinical adoption of AI in radiology by alleviating the collection of clinical imaging data and by improving the performance of AI applications. Key points • Data augmentation generally improves the accuracy and generalization of deep learning models. • Traditional data augmentation does not consider the appearance of imaged structures. • Statistical shape and intensity models (SSIM) synthetically generate variations of imaged structures. • SSIM support novel augmentation approaches, demonstrated with computed tomography bone segmentation. Graphical abstract

Sample Size Effect on Musculoskeletal Segmentation: How Low Can We Go?

“…Furthermore, with advancements in modeling non-linear shape variation and/or eliminating the need for shape correspondence, current model-informed augmentation can even be improved [15][16][17]. Nonetheless, only limited research has explored the potential of shape model-informed methods for data augmentation [18][19][20][21][22].…”