Statistical Techniques For Comparing Measurers And Methods Of Measurement: A Critical Review

Abstract

INTRODUCTIONBiomedical investigators often wish to compare two methods of measurement, usually to compare a new method with an established one. It is an important prerequisite for such studies that the same individual must make the measurements or ratings. Alternatively, investigators may wish to compare the performances of two measurers or raters who are using the same method of measurement, or to evaluate the repeatability of measurements made by the same observer.In clinical and laboratory biomedical science, measurements of a variable are usually made on a continuous scale. Examples of this are measurements of blood pressure, blood gases, lung function and plasma concentrations of a variety of endogenous or exogenous substances. In contrast, categorical scales are used to describe or score attributes by epidemiologists, social scientists, clinicians and, occasionally, laboratory scientists. These categorical scales may be unordered, as in the description of eye colour or taste, or they may be ordered, as in rating scales for cardiovascular functional status, operative (anaesthetic) risk, severity of stroke and any number of ad hoc scales. Somewhere in between are quasi-continuous scales, such as indices of disability, quality of life and so forth, when the range of the scales can be between 20 and 42.Whatever the scale of measurement, there is an unusual consensus among biostatisticians that the goal of making these comparisons should not be to demonstrate agreement, but to detect disagreement or bias. However, biostatisticians disagree, sometimes sharply, on how best to achieve this goal. The purpose of the present review is to describe some of the statistical techniques that can be used to detect bias and to evaluate them critically. CONTINUOUS VARIABLESThis heading is shorthand for variables that are measured on a continuous, or interval, scale. These measurements include distance, weight, concentration, pressure, velocity, temperature, age and so forth. This section deals with three techniques for comparing methods of measurement: (i) regression analysis; (ii) the method of differences; and (iii) correlation. Detecting bias by regression analysisAs a rule, the values obtained by one method of measurement are linearly related to those obtained by another. It seems to follow logically, therefore, that linear regression analysis would be a useful tool for comparing methods of measurement. But what sort of linear regression analysis?The familiar form is least squares of y regression analysis, commonly known as ordinary least squares (OLS) regression. This is what is provided by most computer statistical programs. But, for comparing two methods of measurement, this is the wrong model on two counts. First, under statistical theory, it is an assumption of

Comparative analysis of two FFQ

“…Agreement between both questionnaires, according to the Bland and Altman LOA (18) and ordinary least products regression (19) are shown in Table 1.…”

“…In comparison with earlier studies, Pearson correlation coefficients and paired t tests were used to compare the questionnaires; however, the limited usefulness of these methods is acknowledged. Agreement between both questionnaires was assessed according to the Bland-Altman plots and limits of agreement (LOA) (18) and ordinary least products regression to test for the presence of fixed and proportional bias (19) . Proportional bias exists when there is a relationship between the extent of agreement and the level of intake.…”

Evaluation of air-displacement plethysmography and bioelectrical impedance analysis vs dual-energy X-ray absorptiometry for the assessment of fat-free mass in elderly subjects

“…Predictors identified at bootstrap analysis were entered into multiple regression models with standard errors of coefficients calculated by bootstrap analysis on 1000 random samples of 68 subjects (Gonçalves and White, 2005). Bland and Altman method was used to calculate the limits of agreement between ADP and DXA for the assessment of FFM and Pitman test was used to evaluate proportional bias (Ludbrook, 2002). Bias was defined as the difference between FFM measured by ADP or estimated by BIA and FFM measured by DXA.…”

Left ventricular diastolic function assessment from three‐dimensional three‐directional velocity‐encoded MRI with retrospective valve tracking

“…Multivariate linear regression analysis was performed to identify independent correlation between E/A ratio with Doppler echocardiography and with 3D and 2D VE MRI. Agreement in classifying LV diastolic filling pattern with both VE MRI techniques versus Doppler echocardiography was assessed using kappa (k) weighted statistics (18). Furthermore, sensitivity, specificity, and positive and negative predictive value were obtained to express the clinical value of the techniques when used for classifying the diastolic filling pattern.…”