Use of expert judgment in exposure assessment. Part I. Characterization of personal exposure to benzene

Abstract

This paper presents the results of the first phase of a study, conducted as an element of the National Human Exposure Assessment Survey ( NHEXAS ) , to demonstrate the use of expert subjective judgment elicitation techniques to characterize the magnitude of and uncertainty in environmental exposure to benzene. In decisions about the value of exposure research or of regulatory controls, the characterization of uncertainty can play an influential role. Classical methods for characterizing uncertainty may be sufficient when adequate amounts of relevant data are available. Frequently, however, data are neither abundant nor directly relevant, making it necessary to rely to varying degrees on subjective judgment. Since the 1950s, methods to elicit and quantify subjective judgments have been explored but have rarely been applied to the field of environmental exposure assessment. In this phase of the project, seven experts in benzene exposure assessment were selected through a peer nomination process, participated in a 2 -day workshop, and were interviewed individually to elicit their judgments about the distributions of residential ambient, residential indoor, and personal air benzene concentrations ( 6 -day integrated average ) experienced by both the non -smoking, non -occupationally exposed target and study populations of the US EPA Region V pilot study. Specifically, each expert was asked to characterize, in probabilistic form, the arithmetic means and the 90th percentiles of these distributions. This paper presents the experts' judgments about the concentrations of benzene encountered by the target population. The experts' judgments about levels of benzene in personal air were demonstrative of patterns observed in the judgments about the other distributions. They were in closest agreement about their predictions of the mean; with one exception, their best estimates of the mean fell within 7 -11 g / m 3 although they exhibited striking differences in the degree of uncertainty expressed. Their estimates of the 90th percentile were more varied with the best estimates ranging from 12 to 26 g / m 3 for all but one expert. However, their predictions of the 90th percentile were far more uncertain. The paper demonstrates that coherent subjective judgments can be elicited from exposure assessment scientists and critically examines the challenges and potential benefits of a subjective judgment approach. The results of the second phase of the project, in which measurements from the NHEXAS field study in Region V are used to calibrate the experts' judgments about the benzene exposures in the study population, will be presented in a second paper.

Use of expert judgment in exposure assessment: Part 2. Calibration of expert judgments about personal exposures to benzene

“…Part I of this study (Walker et al, 2001 ) demonstrated that, using principles of good elicitation practices, coherent probability distributions about levels of benzene in ambient, indoor, and personal air in U.S. EPA's Region V could be elicited from experts in this field. In the absence of appropriate measurement data, their probability judgments about the mean and 90th percentiles of benzene in ambient, indoor and personal air in Region V could be used as inputs to discussions about the appropriate level of regulatory concern about current levels of benzene or to decisions about the value and focus of future monitoring programs for benzene.…”

“…The judgments calibrated in this paper are the experts' predictions about the mean and 90th percentile of the residential ambient, residential indoor, and personal air concentrations ( exposure ) in EPA Region V. As discussed in detail in Part I of our study, seven experts knowledgeable about various facets of benzene personal exposure were chosen through a peer-nomination process, attended a group workshop to discuss the project and data relevant to the questions, and participated in a formal elicitation of their individual judgments ( Walker et al, 2001).…”

“…Modest uncertainty distributions with geometric standard deviations of 1.5 were assumed. probabilistic judgment reflects the individual expert's state of knowledge and is therefore unique in some regards, many of the experts defined similar conceptual or ''mental models'' ( Bostrom et al, 1992( Bostrom et al, , 1994 for estimating personal exposures to benzene and relied, to varying degrees, on the same studies of benzene levels in ambient, indoor and personal air ( Walker et al, 2001). Consequently, positive correlations between experts' judgments about these quantities were expected.…”

“…Likewise, numerous investigators have demonstrated the use of subjective methods to characterize quantitatively the state of knowledge about a subject when data are sparse or are not directly relevant. Most recently, the first part of this study (Walker et al, 2001 ) demonstrated how subjective judgment elicitation methods can be applied to the assessment of environmental exposures.…”

“…In this paper, Part II of our investigation into the use of subjective expert judgment in exposure assessment, we report on the calibration of the experts' judgments obtained in Part I of the study ( Walker et al, 2001 ). Using two calibration methods, we compare the experts' probabilistic estimates of the mean and 90th percentile of the distributions of 6 -day averaged residential ambient, residential indoor, and personal air concentrations of benzene for the NHEXAS Region V pilot study population to the actual findings from the NHEXAS study.…”