ROC analysis in patient specific quality assurance

Carlone, Marco; Cruje, Charmainne; Rangel, Alejandra; McCabe, Ryan; Nielsen, Michelle; MacPherson, M

doi:10.1118/1.4795757

Cited by 53 publications

(68 citation statements)

References 37 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Assessment of the performance of different patient‐specific treatment QA acceptance criteria was done by applying receiver operating characteristic (ROC) analysis. While this methodology originally had been used to quantify the value of a diagnostic imaging test, it was also successfully applied to reveal the error detection capabilities and limits for IMRT QA . For both IQM and ArcCHECK QA, the measures of sensitivity (true positive rate) and specificity (true negative rate) were computed for different QA tests at varying decision thresholds.…”

Section: Methodsmentioning

confidence: 99%

Sensitivity of the IQM transmission detector to errors of VMAT plans

et al. 2018

View full text Add to dashboard Cite

Purpose The integral quality monitor (IQM) transmission detector is a wedge‐shaped large area ionization chamber that reports a position‐weighted dose area product for each control point of an IMRT or VMAT plan. In this study, the accuracy of the signal prediction is verified for the Synergy Agility MLC. Tolerance criteria for VMAT plan verification with the IQM were obtained from the observed sensitivity for the detection of incorrectly delivered plans. Methods The predicted IQM signal was compared to the measured signal recorded for a set of 30 VMAT plans for each beam quality of 6 and 10 MV. The system's capability to detect incorrectly delivered plans was tested by measuring altered plans containing small, random deviations. In addition, the observed deviations were related to measurements performed with a second QA phantom. Results The cumulative IQM signal per arc deviated from the respective calculation on average by −0.48% (6 MV) and +0.21% (10 MV) with a standard deviation of 1.08% in both cases, suggesting a 2% warning and 3% action threshold as plan acceptance criteria. This choice was confirmed by the optimum threshold of 2.5% obtained via receiver operating characteristic (ROC) analysis. Reproducibility of individual control points in multiply measured plans was low (on average 7% for 1SD) and thus, segment‐by‐segment comparison was impractical. A suitable criterion to resolve the angular distribution of the plan was identified by binning three to five control points as a running average. While the correlation between IQM signal deviations and gamma passing rates obtained with the ArcCHECK phantom was low for clinical plans, it was apparent for erroneous plans. Binning led to even higher sensitivity to errors. Conclusions The IQM was able to detect induced errors at least as reliable as the standard phantom and showed the potential to be used in pretreatment plan verification to ensure the correct plan transfer and delivery. However, there is no direct correlation between the IQM signal deviation and DVH metrics, so the IQM should be primarily used to screen for errors. Finer diagnostics should then be carried out using a different phantom.

show abstract

Section: Methodsmentioning

confidence: 99%

Sensitivity of the IQM transmission detector to errors of VMAT plans

et al. 2018

View full text Add to dashboard Cite

show abstract

“…Carlone et al . investigated the use of receiver operating characteristic (ROC) methods in order to set tolerance limits for γ evaluations.…”

Section: Review Of Reported Imrt Measured Vs Calculated Agreementmentioning

confidence: 99%

Tolerance limits and methodologies forIMRTmeasurement‐based verificationQA:Recommendations ofAAPMTask Group No. 218

et al. 2018

View full text Add to dashboard Cite

Purpose: Patient-specific IMRT QA measurements are important components of processes designed to identify discrepancies between calculated and delivered radiation doses. Discrepancy tolerance limits are neither well defined nor consistently applied across centers. The AAPM TG-218 report provides a comprehensive review aimed at improving the understanding and consistency of these processes as well as recommendations for methodologies and tolerance limits in patient-specific IMRT QA. Methods: The performance of the dose difference/distance-to-agreement (DTA) and c dose distribution comparison metrics are investigated. Measurement methods are reviewed and followed by a discussion of the pros and cons of each. Methodologies for absolute dose verification are discussed and new IMRT QA verification tools are presented. Literature on the expected or achievable agreement between measurements and calculations for different types of planning and delivery systems are reviewed and analyzed. Tests of vendor implementations of the c verification algorithm employing benchmark cases are presented. Results: Operational shortcomings that can reduce the c tool accuracy and subsequent effectiveness for IMRT QA are described. Practical considerations including spatial resolution, normalization, dose threshold, and data interpretation are discussed. Published data on IMRT QA and the clinical experience of the group members are used to develop guidelines and recommendations on tolerance and action limits for IMRT QA. Steps to check failed IMRT QA plans are outlined. Conclusion: Recommendations on delivery methods, data interpretation, dose normalization, the use of c analysis routines and choice of tolerance limits for IMRT QA are made with focus on detecting differences between calculated and measured doses via the use of robust analysis methods and an in-depth understanding of IMRT verification metrics. The recommendations are intended to improve the IMRT QA process and establish consistent, and comparable IMRT QA criteria among institutions.

show abstract

“…(5) Many studies have tried to find the best cutoff to separate acceptable and unacceptable beams. Carlone et al (6) provided optimal threshold values with introducing random multileaf collimator (MLC) position errors, which can maximize the sensitivity and specificity of the test by ROC analysis. McKenzie et al (7) showed that different IMRT QA devices and QA evaluation methods should have different cutoffs after comparing the result of different IMRT QA devices and QA evaluation methods to a “true” IMRT QA result.…”

Section: Introductionmentioning

confidence: 99%

Are simple IMRT beams more robust against MLC error? Exploring the impact of MLC errors on planar quality assurance and plan quality for different complexity beams

Wang

Jin

Peng

et al. 2016

J Applied Clin Med Phys

View full text Add to dashboard Cite

This study investigated the impact of beam complexities on planar quality assurance and plan quality robustness by introducing MLC errors in intensity‐modulate radiation therapy. Forty patients' planar quality assurance (QA) plans were enrolled in this study, including 20 dynamic MLC (DMLC) IMRT plans and 20 static MLC (SMLC) IMRT plans. The total beam numbers were 150 and 160 for DMLC and SMLC, respectively. Six different magnitudes of MLC errors were introduced to these beams. Gamma pass rates were calculated by comparing error‐free fluence and error‐induced fluence. The plan quality variation was acquired by comparing PTV coverage. Eight complexity scores were calculated based on the beam fluence and the MLC sequence. The complexity scores include fractal dimension, monitor unit, modulation index, fluence map complexity, weighted average of field area, weighted average of field perimeter, and small aperture ratio false(<5cm2 and<50 cm2false). The Spearman's rank correlation coefficient was calculated to analyze the correlation between these scores and gamma pass rate and plan quality variation. For planar QA, the most significant complexity index was fractal dimension for DMLC false(p=−0.40false) and weighted segment area for SMLC false(p=0.27false) at low magnitude MLC error. For plan quality, the most significant complexity index was weighted segment perimeter for DMLC false(p=0.56false) and weighted segment area for SMLC false(p=0.497false) at low magnitude MLC error. The sensitivity of planar QA was weakly associated with the field complexity with low magnitude MLC error, but the plan quality robustness was associated with beam complexity. Plans with simple beams were more robust to MLC error.PACS number(s): 87.55

show abstract

ROC analysis in patient specific quality assurance

Cited by 53 publications

References 37 publications

Sensitivity of the IQM transmission detector to errors of VMAT plans

Sensitivity of the IQM transmission detector to errors of VMAT plans

Tolerance limits and methodologies forIMRTmeasurement‐based verificationQA:Recommendations ofAAPMTask Group No. 218

Are simple IMRT beams more robust against MLC error? Exploring the impact of MLC errors on planar quality assurance and plan quality for different complexity beams

Contact Info

Product

Resources

About