Assessing compatibility of two laboratories: formulations as a statistical hypothesis testing problem

Abstract

A decision problem frequently occurring in metrology is that of compatibility of data obtained by two (or several) different laboratories, methods or instruments. One laboratory can be a prestigious national metrology institute whose data are considered to be a gold standard or a certified reference material interval. When each laboratory presents its results in the form of a coverage interval for the measurand, several statistical approaches to this conformity assessment problem are reviewed including the classical 'equality of means' hypotheses tests. A new compatibility hypothesis is formulated in terms of consistency of laboratory results and compliance with a maximum permitted uncertainty. The power functions of these tests are compared numerically. The Kullback-Leibler information number is suggested as a directional (asymmetric) interchangeability index.

Bayesian hypothesis testing for key comparisons

“…Whether a unilateral degree of freedom differs significantly from zero or not is usually taken as the basis for approving, or disapproving, the uncertainty (and the CMC) quoted by the corresp onding laboratory. Proceeding in such a way corresponds to carrying out a hypothesis test, and a corresponding treatment using means of classical statistics has already been suggested in the context of key comparison data analysis [14,42]. Using classical hypothesis testing does not provide a probability of a considered hypothesis being true in the light of the data.…”

Bias detection and certified reference materials for random measurands

“…Another goal may be to assess lab's compatibility with the certificate which means that the bias ∆ is negligible. It can be argued that compatibility implies that the ratio u / c 2 θ should not be very large [14], but in this work the main focus is on confidence intervals for ∆. The parameters covered by such an interval represent values of ∆ defining a point null hypothesis that cannot be rejected on the basis of given data.…”

“…Here we explore a Bayesian approach to bias checking assuming that the certificate specifies the random measurand's probability distribution. This approach proposed in [14] and elaborated in section 2 raises a few statistical issues with regard to bias estimation. To resolve them, modern higher order asymptotic methods are introduced and implemented in section 4 on the basis of a new maximum likelihood variance estimator obtained in section 3.…”

Compatibility verification of certified reference materials and user measurements

“…It is important to realize that overlapping intervals do not imply that the lab's mean coincides with µ crm . Reference [30] suggests different formulations of compatibility hypothesis in metrology and provides numerical power comparisons of various procedures. In this work we concentrate on the following t-test for two reasons.…”