Deriving Coefficients of Reliability and Agreement for Ratings

Abstract

The paper is concerned with the measurement of internal consistency of rating scales and interviewing schedules, with the assessment of bias between different raters and with coefficients for measuring the degree of agreement between them. Analysis of variance models are first employed, but reference is also made to earlier psychometric techniques and to recent work by Armitage et al. and by Fleiss.

Isolating Suspiciousness from Spectrum-Based Fault Localization Techniques

“…Because most of the formulas used in fault localization are actually brought from other research areas, such as mathematics, data mining, bioinformatics, etc. Among which Scott [21], Geometric Mean (referred to as GMean in the table) [22] and Rogot [23] are introduced from the area of biometrics. And one of the unnamed metrics that is referred to as M in [14] is previously used for classification and clustering [24].…”

Estimating psychiatric morbidity in the community Standardization of the Italian versions of GHQ and CIS

“…Finally a three way analysis of variance without replication was carried out, using as dimensions raters, subjects and symptoms, according to Maxwell and Pilliner (1968); we assumed that raters and symptoms had fixed effects and subjects had random effects, as Mignolli et al 1988) also did. In the analysis, the mean square for the main effect of raters was tested against the first order interaction between raters and subjects; the mean square for the main effect of items was tested against the first order interaction between items and subjects; the mean square for the main effect of subjects and items were tested against the three factor interaction mean square.…”

A model for spectra-based software diagnosis

“…Jaccard and Simple-Matching have also been used in clustering, along with with two unnamed metrics that we refer to as M1 and M2, respectively [Everitt 1978]. In the area of biometrics, various metrics have been introduced: Goodman and Kruskal [1954], Scott [1955], Fleiss [1965], Cohen [1960], Geometric Mean [Maxwell and Pilliner 1968] as well as Arithmetic Mean, Harmonic Mean, Rogot1 and Rogot2 [Rogot and Goldberg 1966]. In the field of clustering using the Self-Organizing Maps (SOM) algorithm various similarity measures or metrics have been introduced and evaluated: Kulczynski1, Kulczynski2, Hamann and Sokal [Lourenco et al 2004].…”