Measurement protocols, random-variable-valued measurements, and response process error: Estimation and inference when sample data are not deterministic

Abstract

Random-variable-valued measurements (RVVMs) are proposed as a new framework for treating measurement processes that generate non-deterministic sample data. They operate by assigning a probability measure to each observed sample instantiation of a global measurement process for some particular random quantity of interest, thus allowing for the explicit quantification of response process error. Common methodologies to date treat only measurement processes that generate fixed values for each sample unit, thus generating full (though possibly inaccurate) information on the random quantity of interest. However, many applied research situations in the non-experimental sciences naturally contain response process error, e.g. when psychologists assess patient agreement with various diagnostic survey items or when conservation biologists perform formal assessments to classify species-at-risk. Ignoring the sample-unit-level uncertainty of response process error in such measurement processes can greatly compromise the quality of resulting inferences. In this paper, a general theory of RVVMs is proposed to handle response process error, and several applications are considered.

Generalized measurement error: Intrinsic and incidental measurement error

“…Specifically, we use the framework of random-variable-valued measurements (RVVMs) introduced in Kroc [ 1 ] to generalize what can be meant by measurement error. This is not a statistical model, and this is not a proposed alternative to current methodologies; it is a probabilistic framework that allows for a generalization of traditional measurement error modelling to encompass novel sources of measurement uncertainty.…”

“…This is not a statistical model, and this is not a proposed alternative to current methodologies; it is a probabilistic framework that allows for a generalization of traditional measurement error modelling to encompass novel sources of measurement uncertainty. In [ 1 ], Kroc’s definition of this framework was imprecise and relied on an untestable measurability assumption for most applications; in this paper, a more precise and rigorous definition is proposed that does not require this assumption. Details appear in the Methods section, but for now, it suffices to state the following.…”

“…These sample measurements, ρ X ( ω 1 ), …, ρ X ( ω n ) are allowed to be random variables. As was shown in Kroc [ 1 ], and as we will review in the next section, such a formulation generalizes the classical sample data situation to allow for uncertainty inherent to the sample measurement process itself; i.e., with this formulation, one may encode measurement uncertainty at the level of the individual sample unit ω ∈ Ω , and such measurement uncertainty is allowed to be potentially unique on each element. This capability is desirable in a variety of applied data situations, and we discuss some of these in the subsequent text, though our main focus is on the mathematics of this generalized observed data framework.…”

“…This capability is desirable in a variety of applied data situations, and we discuss some of these in the subsequent text, though our main focus is on the mathematics of this generalized observed data framework. The reader does not have to be familiar with the content of Kroc [ 1 ], although that paper can be a useful reference for additional example applications of the RVVM framework.…”

“…After establishing the foundational definitions of the framework (Methods section), we will then expand on the work in [ 1 ] in the Results section by defining calibrating conditions that generalize the two most classical kinds of measurement error models, generally attributable to Pearson [ 20 ], Spearman [ 21 ], and Berkson [ 22 ]. We will discuss how the notion of generalized Berkson error in particular mathematicizes what it means to be an expert assessor or rater for a measurement process.…”