Categorization of measures of quality in radiation treatment.

Abstract

187 Background: The Canadian Partnership for Quality Radiotherapy (CPQR) has revised an outdated version of quality assurance (QA) guidelines for use in Canadian radiation treatment (RT) centres. This updated document contains concise, specific quality statements against which program structure and performance can be evaluated. Our objective was to review available international guidelines for quality in RT, categorize them according to a defined taxonomy, and inform a modified Delphi process for further evaluation of the guidelines. Methods: We identified 8 relevant standards documents issued by jurisdictions internationally. Taken together, 454 statements, or groups of statements, describing measures of quality were identified. These disparate quality statements were consolidated into a limited number of manageable groups using an appropriate classification scheme. A decision tree with two major categories was developed. Category 1 was based on the classification proposed by Donabedian as to whether the statement addressed a structure, process, outcome, or other. Category 2 addressed whether the statement referred to activities directed towards the organization, patients, staff, equipment/clinical processes, or other. Five reviewers assigned each of the 454 statements independently to one of these 20 (4 x 5) category decision tree endpoints. Results: A free marginal kappa analysis of agreement between the 5 reviewers in all 20 endpoints yielded a value of 0.38 which is fair agreement. Restricting analysis to Category 1, there is better agreement with a kappa of 0.54. Alternatively, 3 or more of the 5 reviewers agreed on their assignments 93% of the time for Category 1 and 76% for Categories 1 and 2. There were 290 statements with less than 100% agreement in Category 1 extracted for a second mapping using additional descriptive qualifiers. Subsequently, the overall free-marginal kappa increased to 0.71, considered substantial agreement. Conclusions: Kappa values varied among the 8 documents reviewed, possibly indicating differences in clarity of the descriptions. The 454 standards identified have been categorized and provide a manageable input to our ongoing effort to maintain appropriate, up-to-date Canadian quality guidelines for RT.