A deliberative method for ranking risks was evaluated in a study involving 218 risk managers. Both holistic and multiattribute procedures were used to assess individual and group rankings of health and safety risks facing students at a fictitious middle school. Consistency between the rankings that emerged from these two procedures was reasonably high for individuals and for groups, suggesting that these procedures capture an underlying construct of riskiness. Participants reported high levels of satisfaction with their groups' decision-making processes and the resulting rankings, and these reports were corroborated by regression analyses. Risk rankings were similar across individuals and groups, even though individuals and groups did not always agree on the relative importance of risk attributes. Lower consistency between the risk rankings from the holistic and multiattribute procedures and lower agreement among individuals and groups regarding these rankings were observed for a set of high-variance risks. Nonetheless, the generally high levels of consistency, satisfaction, and agreement suggest that this deliberative method is capable of producing risk rankings that can serve as informative inputs to public risk-management decision making.

Risk ranking offers a potentially powerful means for gathering public input to help set risk-management priorities. In most rankings conducted to date, the categories and attributes used to describe the risks have varied widely, the materials and procedures have not been designed to facilitate comparisons among risks on all important attributes, and the validity and reproducibility of the resulting rankings have not been assessed. To address these needs, a risk-ranking method was developed in which risk experts define and categorize the risks to be ranked, identify the relevant risk attributes, and characterize the risks in a set of standardized risk summary sheets, which are then used by lay or other groups in structured ranking exercises. To evaluate this method, a test bed involving 22 health and safety risks in a fictitious middle school was created. This article provides an overview of the risk-ranking method and describes the challenges faced in designing the middle school test bed. A companion article in this issue reports on the validity of the ranking procedures and the level of agreement among risk managers regarding ranking of risks and attributes.

Any practical process of risk ranking must group hazards into a manageable number of categories. Defining such categories requires value choices that can have important implications for the rankings that result. Most risk-management organizations will find it useful to begin defining categories in terms of environmental loadings or initiating events. However, the resulting categories typically need to be modified in light of other considerations. Risk-ranking projects can benefit from considering several alternative categorization strategies and drawing upon elements of each in developing their final categorization of risks. In principle, conducting multiple ranking exercises by using different categorizations could be interesting and useful. In practice, agencies are unlikely to have either the resources or patience to do this, but other groups in society might. Done well, such additional independent rankings could add valuable inputs to democratic risk-management decision making.

This article reports an extension of the Carnegie Mellon risk-ranking method to incorporate ecological risks and their attributes. On the basis of earlier risk-perception studies, we identified a set of 20 relevant attributes for describing health, safety, and environmental hazards in standardized risk summary sheets. In a series of three ranking sessions, 23 laypeople ranked 10 such hazards in a fictional Midwestern U.S. county using both holistic and multiattribute ranking procedures. Results were consistent with those from previous studies involving only health and safety hazards, providing additional evidence for the validity of the method and the replicability of the resulting rankings. Holistic and multiattribute risk rankings were reasonably consistent both for individuals and for groups. Participants reported that they were satisfied with the procedures and results, and indicated their support for using the method to advise real-world risk-management decisions. Agreement among participants increased over the course of the exercise, perhaps because the materials and deliberations helped participants to correct their misconceptions and clarify their values. Overall, health and safety attributes were judged more important than environmental attributes. However, the overlap between the importance rankings of these two sets of attributes suggests that some information about environmental impacts is important to participants' judgments in comparative risk-assessment tasks.

No abstract