Critical issues concerning the assessment sets in agreement assessment

Abstract

Since Bland and Altman's pioneering work, the assessment set has been an important constituent of agreement assessment. This article explores critical issues concerning the assessment set. First, we point out the fundamental differences between assessment approaches based on point estimation and those based on hypothesis testing related to the assessment set, and further present how the related disciplines choose between the two. Second, we argue that an assessment set with minimum volume should be preferred, as in the comparison of confidence intervals. Finally, we discuss assessment sets for the assessment of multiple methods, and propose two methods for the construction of assessment sets. Numerical examples further establish the applicability of these approaches.