Large scale crowdsourced radiotherapy segmentations across a variety of cancer anatomic sites

Abstract

Clinician generated segmentation of tumor and healthy tissue regions of interest (ROIs) on medical images is crucial for radiotherapy. However, interobserver segmentation variability has long been considered a significant detriment to the implementation of high-quality and consistent radiotherapy dose delivery. This has prompted the increasing development of automated segmentation approaches. However, extant segmentation datasets typically only provide segmentations generated by a limited number of annotators with varying, and often unspecified, levels of expertise. In this data descriptor, numerous clinician annotators manually generated segmentations for ROIs on computed tomography images across a variety of cancer sites (breast, sarcoma, head and neck, gynecologic, gastrointestinal; one patient per cancer site) for the Contouring Collaborative for Consensus in Radiation Oncology challenge. In total, over 200 annotators (experts and non-experts) contributed using a standardized annotation platform (ProKnow). Subsequently, we converted Digital Imaging and Communications in Medicine data into Neuroimaging Informatics Technology Initiative format with standardized nomenclature for ease of use. In addition, we generated consensus segmentations for experts and non-experts using the Simultaneous Truth and Performance Level Estimation method. These standardized, structured, and easily accessible data are a valuable resource for systematically studying variability in segmentation applications.

Associations Between Radiation Oncologist Demographic Factors and Segmentation Similarity Benchmarks: Insights From a Crowd-Sourced Challenge Using Bayesian Estimation

“…Greater details on the publicly available C3RO data set can be found in the corresponding data descriptor. 7 Self-reported demographic variables of interest from the participants were initially collected through an intake survey performed on REDCap. 8 Informed by previous research, 9 , 10 various demographic variables were collected for physicians in this study ( Table 1 ).…”

“…To ensure that metrics were comparable across ROIs, metrics were stratified into binary groups on the basis of previously established ROI-specific expert-derived IOV cutoffs—cutoffs were calculated as the median of pairwise metric values for all available expert segmentations. 7 Namely, if the metric for a given ROI was greater than or equal to the ROI-specific expert IOV, it was classified as 1, otherwise, 0 ( Fig 1 ). Finally, for each ROI, we calculated the percentage of observers who were able to cross the expert IOV cutoff.…”

OIMHS: An Optical Coherence Tomography Image Dataset Based on Macular Hole Manual Segmentation

“…In order to evaluate the raters’ capacity, the STAPLE algorithm was introduced 13 . The primary objective in integrating the STAPLE algorithm was to comprehensively assess the capacity and reliability of the four raters involved in the labeling process 14 , 15 . By leveraging this algorithm, we aimed to obtain a robust gold standard annotation encompassing the collective input of three ophthalmologists and one expert.…”

Essential parameters needed for a U-Net-based segmentation of individual bones on planning CT images in the head and neck region using limited datasets for radiotherapy application

“…Therefore, further validation and evaluation are needed for standalone deep learning approaches before their introduction into clinical routine. On the other hand, the field of radiotherapy is highly marked with a lack of datasets even with the availability of public datasets such as the StructSeg2019 segmentation for radiotherapy planning challenge 2019 (Wahid et al 2023). Hence, this need raises concern on the essential parameters needed by deep learning models to yield an accurate segmentation of individual bones on planning CT images with limited training datasets.…”