Abstract. An internal audit in quality management system based on ISO 9001:2000 is an important process that continually improves the effectiveness and efficiency of a quality management system. The reliability of assessments of an internal audit based on ISO 9001:2000 is an issue that continues to demand the attention of researchers in this field. One element of reliability is the extent to which different teams assessing the same project produce similar ratings when presented with the same evidence. Cohen's Kappa coefficient has been the most popular measure of reliability for measuring the degree of interrater agreement beyond chance. However, the application of the Kappa coefficient may be biased downwards due to two Kappa "paradoxes." To compensate for the deficiency of the Kappa coefficient, we additionally use a simpler index of observed agreement between two teams of auditors. Using Cohen's Kappa coefficient and the observed agreement, this paper presents some results from two assessments between two teams conducted during internal audits in Electronics and Telecommunications Research Institute in Korea. In each of these assessments, two independent teams performed the same ratings. The results indicate that in general there is at least moderate agreement between the two teams and the extent of agreement between the two teams is substantial.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.