Expert Judgement and Expert Disagreement

Mumpower, Jeryl L.; Stewart, Thomas R.

doi:10.1080/135467896394500

Cited by 76 publications

(75 citation statements)

References 50 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The triple-system design, which is an expansion of the lens model, involves a task situation and two people making use of the same probabilistic cues. This has enabled the study of interpersonal learning (Earle, 1973;Hammond, 1972; and interpersonal conflict (B. Brehmer, 1976;Hammond, 1965Hammond, , 1973Hammond, Todd, Wilkins, & Mitchell, 1966;Mumpower & Stewart, 1996). Finally, the N-system design involves more than two people and may or may not include an outcome criterion, and so it enables a study of group judgment (e.g., Rohrbaugh, 1988).…”

Section: Social Judgment Theory Policy Capturing and The Importancementioning

confidence: 99%

The Role of Representative Design in an Ecological Approach to Cognition.

Dhami¹,

Hertwig²,

Hoffrage³

2004

Psychological Bulletin

371

285

View full text Add to dashboard Cite

Egon Brunswik argued that psychological processes are adapted to environmental properties. He proposed the method of representative design to capture these processes and advocated that psychology be a science of organism-environment relations. Representative design involves randomly sampling stimuli from the environment or creating stimuli in which environmental properties are preserved. This departs from systematic design. The authors review the development of representative design, examine its use in judgment and decision-making research, and demonstrate the effect of design on research findings. They suggest that some of the practical difficulties associated with representative design may be overcome with modern technologies. The importance of representative design in psychology and the implications of this method for ecological approaches to cognition are discussed.

show abstract

Section: Social Judgment Theory Policy Capturing and The Importancementioning

confidence: 99%

The Role of Representative Design in an Ecological Approach to Cognition.

Dhami¹,

Hertwig²,

Hoffrage³

2004

Psychological Bulletin

371

285

View full text Add to dashboard Cite

show abstract

“…• the ability of the experts to encode their knowledge probabilistically [15,87,134,176] and their potential for overconfidence [14,27,69,111,177]; • the correlation between expert judgements that arises from their shared knowledge and common professional backgrounds [16,55,128,156]; • the correlation between the experts' judgements and the DM's own judgements [54,55];…”

Section: The Expert Problem: a Brief Survey Of Developmentsmentioning

confidence: 99%

“…For some discussion of these topics, see Cooke and Goossens [34], Mumpower and Stewart [128] and Ortiz et al [136].…”

Section: Introductionmentioning

confidence: 98%

Aggregating expert judgement

French

2011

RACSAM

View full text Add to dashboard Cite

In a paper written some 25 years ago, I distinguished three contexts in which one might wish to combine expert judgements of uncertainty: the expert problem, the group decision problem and the textbook problem. Over the intervening years much has been written on the first two, which have the focus of a single decision context, but little on the third, though the closely related field of meta-analysis has developed considerably. With many developments in internet technology, particularly in relation to interactivity and communication, the textbook problem is gaining in importance since data and expert judgements can be made available over the web to be used by many different individuals to shape their own beliefs in many different contexts. Moreover, applications such as web-based decision support, e-participation and e-democracy are making algorithmic 'solutions' to the group decision problem attractive, despite many results showing we know that such solutions are, at best, rare and, at worst, illusory. In this paper I survey developments since my earlier paper and note some unresolved issues. Then I turn to how expert judgement might be used within web-based group decision support, as well as in e-participation and e-democracy contexts. The latter points to a growing importance of the textbook problem and suggests that Cooke's principles for scientific reporting of expert judgement studies may need enhancing for such studies to be used by a wider audience.

show abstract

“…Expert knowledge is typically considered to be the best source of information, especially in unfamiliar situations where definitive data is not available (McBride & Burgman, 2012;Mumpower & Stewart, 1996). However, relying on expert opinion can be problematic.…”

Section: Chapter 1 Expert Decisionsmentioning

confidence: 99%

“…Although there can be many reasons for expert disagreement, it is often simply attributable to the different ways in which individuals think about a problem. Novel scenarios can facilitate varied judgments and conclusions from even the most competent and critical experts (Mumpower & Stewart, 1996).…”

Section: Chapter 1 Expert Decisionsmentioning

confidence: 99%

Resolving contentious hospital observation chart design decisions using a behavioural experimental approach

Christofidis¹

View full text Add to dashboard Cite

A new paper-based hospital observation chart has been developed using human factors design principles. This novel design, compared to previous charts, yielded fewer errors and faster response times in chart-users' detection of patient physiological deterioration compared to other Australian observation charts that were in use at the time. In recent clinical studies, the chart has also been associated with an 11% mortality reduction amongst intensive care unit admissions, as well as a 45% reduction in the incidence of cardiac arrests. However, there are a number of points of contention as to whether this design can be regarded as best practice. First, it is unclear whether the chart offers performance benefits to users highly experienced with alternative chart designs. Second, clinicians have questioned particular features that were designed to help users detect abnormal vital sign observations. For example, there is a dispute as to whether blood pressure and heart rate graphs should be presented as separate or overlapping plots. Third, disagreement surrounds the optimal design layout to facilitate users' calculation of summary scores that represent the physiological state of a patient. In the absence of expert consensus, this thesis sought to address each of these points of contention using behavioural experiments. In general, findings supported the design choices associated with the new observation chart. Specifically, in relation to the detection of abnormal observations, it was found that (1) even users experienced with alternative chart designs performed better with the new chart; (2) blood pressure and heart rate were better presented as separate graphs (even for chart-users who preferred plots that overlap); and (3) users' performance with drawn-dot observations, an integrated colour-based scoring-system, and grouped scoring-rows was consistent with apriori human factors design principles. One design aspect of the new chart was not supported: users were found to be less accurate calculating patient deterioration summary scores when the design involved recording interim steps in the calculation. Overall, it is argued that these experiments demonstrate the value of using behavioural experiments to assess best design practice, rather than relying solely on expert opinion. 2

show abstract

Expert Judgement and Expert Disagreement

Cited by 76 publications

References 50 publications

The Role of Representative Design in an Ecological Approach to Cognition.

The Role of Representative Design in an Ecological Approach to Cognition.

Aggregating expert judgement

Resolving contentious hospital observation chart design decisions using a behavioural experimental approach

Contact Info

Product

Resources

About