Quality assessment of ordinal scale reproducibility: log-linear models provided useful information on scale structure

Assessing the reliability of four severity scales depicting skin ageing features

“…In addition, as recently pointed out by Valet et al. , 22 these results could also indicate that the kappa statistic may indicate an almost perfect degree of agreement, even when a major scale defect exists.…”

Reliability and agreement studies: a guide for clinical investigators

“…Other approaches are log-linear models, which examine the association of cell counts on the level of categories (eg, observers) and models’ interaction patterns and associations 26 46 47. Similar to the latent class models, there are a handful of examples using log-linear models in GI reliability 16 48. Again, STATA provides commands for latent class models (gllamm) and log-linear models (ipf).…”

Validation of digital photographic reference scales for evaluating facial aging signs

“…Evaluation of distinguishability provides valuable information on the structure of a scale . This method adds another dimension to testing the validity of ordinal scales by highlighting more precisely where they can be improved .…”

“…This approach permitted us to refine the scales by either pooling adjacent grades or replacing representative photos to improve the accuracy. Indeed, an equal and high DD value between all adjacent categories of the scale would clearly lead to a lower variability within ratings and hence a better agreement between them …”