Automatic evaluation of contours in radiotherapy planning utilising conformity indices and machine learning

Abstract

Background and purpose: Peer-review of Target Volume (TV) and Organ at Risk (OAR) contours in radiotherapy planning are typically conducted visually; this can be time consuming and subject to interobserver variation. This study investigated automatic evaluation of contouring using conformity indices and supervised machine learning. Methods: A total of 393 contours from 253 Stereotactic Ablative Body Radiotherapy (SABR) benchmark cases (adrenal gland, liver, pelvic lymph node and spine), delineated by 132 clinicians from 25 centres, were visually evaluated for conformity against gold standard contours. Contours were scored as "pass" or "fail" on visual peer review and six Conformity Indices (CIs) were applied. CI values were mapped to pass/fail scores for each contour and used to train supervised machine learning models. A 5-fold cross validation method was employed to determine the predictive accuracies of each model. Results: The stomach structure produced models with the highest predictive accuracy overall (96% using Support Vector Machine and Ensemble models), whilst the liver GTV produced models with the lowest predictive accuracy (76% using Logistic Regression). Predictive accuracies across all models ranged from 68-96% (68-87% for TV and 71-96% for OARs). Conclusions: Although a final visual review by an experienced clinician is still required, the automatic contour evaluation method could reduce the time for benchmark case reviews by identifying gross contouring errors. This method could be successfully implemented to support departmental training and the continuous assessment of outlining for clinical staff in the peer-review process, to reduce interobserver variability in contouring and improve interpretation of radiological anatomy.

Applications of artificial intelligence in stereotactic body radiation therapy

“…An interesting approach was developed by Terparia et al (2020), who built an automated binary reviewer based on conformity indices and supervised ML, to categorise contours as pass or fail when compared to a Gold Standard. Supervised models (Decision Trees, Logistic Regression, SVMs, K-nearest neighbours (KNN) and Ensemble methods) were trained to use conformity indices (e.g.…”

Radiotherapy Quality Assurance in the PORTEC-3 (TROG 08.04) Trial

“…Although QART for the ANZ Gynaecologic Oncology Group cases was carried out for 74/122 patients (61%), only a limited number of cases were assessed in the international post-treatment QART (72/560 patients or 13%) due to the retrospective introduction, workload and cost limitations, which is a limitation to our review. This raises the possibility of improving QART for future trials using automated atlases [19] or algorithms that could quickly identify major contour variations and provide feedback. It would also minimise the interobserver variability noted in manual review.…”

Machine Learning Meets Cancer

“…Finally, the remaining studies were devoted to ML-based gene expression analysis and had an accuracy ranging from 80% to 93%. The study in [ 62 ] examined automatic contour scoring using fit indices and supervised machine learning. The issue is relevant because the peer review of the target volume (TV) and organ at risk (OAR) contours during radiation therapy planning is conducted visually and can be time-consuming, and the result can vary greatly among observers.…”