Quantitative assessment of target delineation variability for thymic cancers: agreement evaluation of a prospective segmentation challenge

Abstract

Objectives We sought to quantitatively determine the inter-observer variability of expert radiotherapy target-volume delineation for thymic cancers, as part of a larger effort to develop an expert-consensus contouring atlas. Methods A pilot dataset was created consisting of a standardized case presentation with pre- and post-operative DICOM CT image sets from a single patient with Masaoka-Koga Stage III thymoma. Expert thoracic radiation oncologists delineated tumor targets on the pre- and post-operative scans as they would for a definitive and adjuvant case, respectively. Respondents completed a survey including recommended dose prescription and target volume margins for definitive and post-operative scenarios. Inter-observer variability was analyzed quantitatively with Warfield's simultaneous truth, performance level estimation (STAPLE) algorithm and Dice similarity coefficient (DSC). Results Seven users completed contouring for definitive and adjuvant cases; of these, 5 completed online surveys. Segmentation performance was assessed, with high mean±SD STAPLE-estimated segmentation sensitivity for definitive case GTV and CTV at 0.77 and 0.80, respectively, and post-operative CTV sensitivity of 0.55; all volumes had specificity of ≥0.99. Inter-observer agreement was markedly higher for the definitive target volumes, with mean±SD DSC of 0.88±0.03 and 0.89±0.04 for GTV and CTV respectively, compared to post-op CTV DSC of 0.69±0.06 (Kruskal-Wallis p<0.01. Conclusion Expert agreement for definitive case volumes was exceptionally high, though significantly lower agreement was noted post-operatively. Technique and dose prescription between experts was substantively consistent, and these preliminary results will be utilized to create an expert-consensus contouring atlas to aid the non-expert radiation oncologist in the planning of these challenging, rare tumors.

An in-silico quality assurance study of contouring target volumes in thoracic tumors within a cooperative group setting

“…Our work follows previous SWOG efforts made to address inter-observer variability across different organs sites [9] . In previous work, we showed that educational intervention with a SWOG-approved consensus atlas reduced inter-observer variability in rectal cancer target volume delineation [7] .…”

Optimal delineation of the clinical target volume for thymomas in the post-resection setting: A multi-center study

“…29 In thymomas, only inter-observer variability between RTOs has been analyzed. 11 Currently, a lack of a standardized protocol for PORT in thymomas leads to only brief advice in international guidelines. 5,8 These results support our findings that a protocol and further research in optimal delineation of CTVs for thymomas in the post-resection setting is necessary.…”

“…5,9,10 Accurate delineation of tumor volumes is a time-consuming and crucial step in radiotherapy, but it is also the most susceptible to human error. 11 Radiation oncologists (RTOs) are often dependent on descriptive language used by other specialists, such as surgeons and radiologists. 12 Computed tomography (CT)-based planning is used to determine the clinical target volume (CTV).…”

A dose based approach for evaluation of inter-observer variations in target delineation

“…Substantial inter-observer variations in target delineation have been presented in a number of previous studies [1][2][3][4][5][6][7][8][9]. The variations can be due to differences in interpretation of the diagnostic material, ambiguities in treatment protocols, lack of guidelines and/or inadequate, differences in local policies, the availability and use of multi-modality imaging, the subjective assessment of disease dissemination and/or the individual training and experience of the radiation oncologists.…”