Medicine in words and numbers: a cross-sectional survey comparing probability assessment scales

Abstract

Background: In the complex domain of medical decision making, reasoning under uncertainty can benefit from supporting tools. Automated decision support tools often build upon mathematical models, such as Bayesian networks. These networks require probabilities which often have to be assessed by experts in the domain of application. Probability response scales can be used to support the assessment process. We compare assessments obtained with different types of response scale.Methods: General practitioners (GPs) gave assessments on and preferences for three different probability response scales: a numerical scale, a scale with only verbal labels, and a combined verbalnumerical scale we had designed ourselves. Standard analyses of variance were performed.Results: No differences in assessments over the three response scales were found. Preferences for type of scale differed: the less experienced GPs preferred the verbal scale, the most experienced preferred the numerical scale, with the groups in between having a preference for the combined verbal-numerical scale. Conclusion:We conclude that all three response scales are equally suitable for supporting probability assessment. The combined verbal-numerical scale is a good choice for aiding the process, since it offers numerical labels to those who prefer numbers and verbal labels to those who prefer words, and accommodates both more and less experienced professionals.

Assessing Diagnostic Expertise of Counselors Using the Cochran–Weiss–Shanteau (CWS) Index

“…Short vignettes (see example below) were shown on the computer screen, and participants were asked to indicate their judgment of the probability that the patient described in each vignette suffers from major depressive disorder. In a slight departure from the usual elicitation, we did not ask our participants for a precise probability score but for a probability interval, because we believe this to be more ecologically valid than asking for a precise number (Renooij & Witteman, 1999; Witteman, Renooij, & Koele, 2007). It also reflects the inherent subjectivity of the judgments and the fact that there are no correct answers.…”

An English-Language Argumentation Interface for Explanation Generation with Markov Decision Processes in the Domain of Academic Advising

“…A verbal scale has inherent problems. In medical decision making, Witteman et al [2007] found that experienced doctors were more confident using a verbal, rather than numeric, scale. Renooij [2001] reports large inter-subject variability of the numerical values assigned to verbal expressions.…”

Effective communication of uncertainty in the IPCC reports

“…The dual approach provides more information, facilitates similar interpretation of the phrases, and caters to a broad and heterogeneous audience with various levels of expertise and preferences (see Witteman and Renooij 2003;Witteman et al 2007). Our results show that supplementing verbal terms with numerical boundaries improved considerably the quality of communication.…”