Scalable radiotherapy data curation infrastructure for deep-learning based autosegmentation of organs-at-risk: A case study in head and neck cancer

Abstract

In this era of patient-centered, outcomes-driven and adaptive radiotherapy, deep learning is now being successfully applied to tackle imaging-related workflow bottlenecks such as autosegmentation and dose planning. These applications typically require supervised learning approaches enabled by relatively large, curated radiotherapy datasets which are highly reflective of the contemporary standard of care. However, little has been previously published describing technical infrastructure, recommendations, methods or standards for radiotherapy dataset curation in a holistic fashion. Our radiation oncology department has recently embarked on a large-scale project in partnership with an external partner to develop deep-learning-based tools to assist with our radiotherapy workflow, beginning with autosegmentation of organs-at-risk. This project will require thousands of carefully curated radiotherapy datasets comprising all body sites we routinely treat with radiotherapy. Given such a large project scope, we have approached the need for dataset curation rigorously, with an aim towards building infrastructure that is compatible with efficiency, automation and scalability. Focusing on our first use-case pertaining to head and neck cancer, we describe our developed infrastructure and novel methods applied to radiotherapy dataset curation, inclusive of personnel and workflow organization, dataset selection, expert organ-at-risk segmentation, quality assurance, patient de-identification, data archival and transfer. Over the course of approximately 13 months, our expert multidisciplinary team generated 490 curated head and neck radiotherapy datasets. This task required approximately 6000 human-expert hours in total (not including planning and infrastructure development time). This infrastructure continues to evolve and will support ongoing and future project efforts.

Deep Learning-Based Segmentation of Head and Neck Organs-at-Risk with Clinical Partially Labeled Data

“…Performance boost after data curation was evident in Figure 3 , supporting the need for homogeneity in the segmentation protocol as also seen in previous works [ 30 , 31 ]. This problem was solved in most studies by requesting a single radiation oncologist to perform or review all the ground truth segmentations.…”

“…Several factors still limit the clinical implementation of DL-based auto-segmentation [ 6 , 9 , 28 ], including a lack of standardization of contouring protocols [ 10 , 29 ], trust among the users, and limited availability of large, labeled databases. Ideally, to train a robust DL segmentation network, an extensive database of patient CT images would need to be labeled, reviewed, and curated by several experts following the same delineation guidelines [ 7 , 30 ]. A second labeled database would allow for external validation.…”

“…These data can also be exploited with DL and, furthermore, used to optimize individual treatment for patients in an adaptive RT framework [ 5 ]. Curating this data may be necessary to ensure homogeneous learning [ 8 , 13 , 30 , 31 ].…”

Validation of clinical acceptability of deep-learning-based automated segmentation of organs-at-risk for head-and-neck radiotherapy treatment planning

“…One limitation is that none of the reported models provides a comprehensive set of all recommended OARs for HN cancer ( 20 ). For instance, the brachial plexus (BP) is often not included in the model, despite having both an important role in treatment planning and generally requiring substantial time to contour ( 21 ). In addition, the segmentations produced by these models generally require substantial manual edits of multiple OARs in order to be accurate enough for treatment planning.…”

“…At our institution, all OAR segmentation is governed by a detailed set of institutional standards, primarily based on international consensus guidelines ( 20 ). Using these standards, two of the authors (both HN-expert ROs) were given protected time away from clinical responsibilities to contour on retrospectively-collected patient datasets, and without the time constraints experienced during daily clinical practice, spending an average of more than 11 hours per patient dataset ( 21 ) (exceeding the typical amount of time available for a clinical case). This effort resulted in a consistent “gold standard” (GS) dataset that best reflects the international consensus and institutional standards for 490 retrospectively-identified patients ( 21 ).…”