Caution! Warnings as a Useless Countermeasure to Reduce Overconfidence? An Experimental Evaluation in Light of Enhanced and Dynamic Warning Designs

Schall, Dominik; Doll, Dominik; Mohnen, Alwine

doi:10.1002/bdm.1946

Cited by 9 publications

(7 citation statements)

References 40 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Concerning debiasing by means of improving students’ knowledge, one might think intuitively that warning people about possible tendencies to overestimate performance might be a helpful countermeasure. While former research however had argued that such warnings do not help to reduce overconfidence (Plous, ), Schall, Doll, and Mohnen () found that repeated (dynamic) warnings do have the potential to reduce overestimation. The question remains whether repeated warnings are the method of choice for the university context.…”

Section: Resultsmentioning

confidence: 99%

“I Thought I Did Much Better”—Overconfidence in University Exams

Wüst

Beck

2018

Decision Sci J Innov Edu

View full text Add to dashboard Cite

Students tend to be overconfident about their future performance in university exams, which can lead to demotivation when faced with unexpectedly bad results. We hypothesized that overconfidence is partly a personality trait but also partly results from students’ insufficient knowledge of what is expected of them in university exams. It follows that this should decrease as the students learn what is really required. In a longitudinal study with 441 students from a southern German university, we asked students to solve intelligence test questions and to make estimates of their own performances. Additionally, students estimated their performance in a forthcoming maths and/or a descriptive statistics exam. Then shortly after the exam was taken they indicated their planned and effective hours of study. As expected, students did not, in the mean, overestimate their performance on intelligence test questions. Women were more likely than men to underestimate their performance. Over‐ or underestimation in intelligence test questions did, however, predict misestimation in the mathematics and the statistics exams. Also, further correlations of misestimation between the statistics and a financial mathematics exam could be found—as such misestimation can be interpreted to be a stable trait. Students with an "Abitur" qualification were more likely to underestimate their performance, while students with other preuniversity qualifications tended to overestimate their performance. No effect of personality factors was found. Against our expectations, students who had overestimated themselves later indicated a higher number of hours studied than other students. We suggest different strategies by which students and universities can actively counter overestimation.

show abstract

Section: Resultsmentioning

confidence: 99%

“I Thought I Did Much Better”—Overconfidence in University Exams

Wüst

Beck

2018

Decision Sci J Innov Edu

View full text Add to dashboard Cite

show abstract

“…Some studies have examined the effect of dynamic features in signs on behavioral compliance during a work-related task and emergency [3,[58][59][60]. The findings suggested that dynamic presentations produced higher compliance than static presentations [58,[61][62][63][64][65] mainly because of some features that make them more noticeable. This is in line with attention theory [66].…”

Section: Related Workmentioning

confidence: 99%

Gamification and Hazard Communication in Virtual Reality: A Qualitative Study

Cavalcanti

Valls

Contero

et al. 2021

Sensors

View full text Add to dashboard Cite

An effective warning attracts attention, elicits knowledge, and enables compliance behavior. Game mechanics, which are directly linked to human desires, stand out as training, evaluation, and improvement tools. Immersive virtual reality (VR) facilitates training without risk to participants, evaluates the impact of an incorrect action/decision, and creates a smart training environment. The present study analyzes the user experience in a gamified virtual environment of risks using the HTC Vive head-mounted display. The game was developed in the Unreal game engine and consisted of a walk-through maze composed of evident dangers and different signaling variables while user action data were recorded. To demonstrate which aspects provide better interaction, experience, perception and memory, three different warning configurations (dynamic, static and smart) and two different levels of danger (low and high) were presented. To properly assess the impact of the experience, we conducted a survey about personality and knowledge before and after using the game. We proceeded with the qualitative approach by using questions in a bipolar laddering assessment that was compared with the recorded data during the game. The findings indicate that when users are engaged in VR, they tend to test the consequences of their actions rather than maintaining safety. The results also reveal that textual signal variables are not accessed when users are faced with the stress factor of time. Progress is needed in implementing new technologies for warnings and advance notifications to improve the evaluation of human behavior in virtual environments of high-risk surroundings.

show abstract

“…by stating that many people make their confidence intervals too narrow) is largely ineffective. 143,152,153 However, in-depth training on the nature of biases and strategies for avoiding them has been found to be more effective. When biases occur as a result of experts not being familiar with rules for using and expressing probabilities, training on how to do so can reduce errors.…”

Section: Bias Warnings and Trainingmentioning

confidence: 99%

Developing a reference protocol for structured expert elicitation in health-care decision-making: a mixed-methods study

Bojke

Soares

Claxton

et al. 2021

Health Technol Assess

View full text Add to dashboard Cite

Background Many decisions in health care aim to maximise health, requiring judgements about interventions that may have higher health effects but potentially incur additional costs (cost-effectiveness framework). The evidence used to establish cost-effectiveness is typically uncertain and it is important that this uncertainty is characterised. In situations in which evidence is uncertain, the experience of experts is essential. The process by which the beliefs of experts can be formally collected in a quantitative manner is structured expert elicitation. There is heterogeneity in the existing methodology used in health-care decision-making. A number of guidelines are available for structured expert elicitation; however, it is not clear if any of these are appropriate for health-care decision-making. Objectives The overall aim was to establish a protocol for structured expert elicitation to inform health-care decision-making. The objectives are to (1) provide clarity on methods for collecting and using experts’ judgements, (2) consider when alternative methodology may be required in particular contexts, (3) establish preferred approaches for elicitation on a range of parameters, (4) determine which elicitation methods allow experts to express uncertainty and (5) determine the usefulness of the reference protocol developed. Methods A mixed-methods approach was used: systemic review, targeted searches, experimental work and narrative synthesis. A review of the existing guidelines for structured expert elicitation was conducted. This identified the approaches used in existing guidelines (the ‘choices’) and determined if dominant approaches exist. Targeted review searches were conducted for selection of experts, level of elicitation, fitting and aggregation, assessing accuracy of judgements and heuristics and biases. To sift through the available choices, a set of principles that underpin the use of structured expert elicitation in health-care decision-making was defined using evidence generated from the targeted searches, quantities to elicit experimental evidence and consideration of constraints in health-care decision-making. These principles, including fitness for purpose and reflecting individual expert uncertainty, were applied to the set of choices to establish a reference protocol. An applied evaluation of the developed reference protocol was also undertaken. Results For many elements of structured expert elicitation, there was a lack of consistency across the existing guidelines. In almost all choices, there was a lack of empirical evidence supporting recommendations, and in some circumstances the principles are unable to provide sufficient justification for discounting particular choices. It is possible to define reference methods for health technology assessment. These include a focus on gathering experts with substantive skills, eliciting observable quantities and individual elicitation of beliefs. Additional considerations are required for decision-makers outside health technology assessment, for example at a local level, or for early technologies. Access to experts may be limited and in some circumstances group discussion may be needed to generate a distribution. Limitations The major limitation of the work conducted here lies not in the methods employed in the current work but in the evidence available from the wider literature relating to how appropriate particular methodological choices are. Conclusions The reference protocol is flexible in many choices. This may be a useful characteristic, as it is possible to apply this reference protocol across different settings. Further applied studies, which use the choices specified in this reference protocol, are required. Funding This project was funded by the NIHR Health Technology Assessment programme and will be published in full in Health Technology Assessment; Vol. 25, No. 37. See the NIHR Journals Library website for further project information. This work was also funded by the Medical Research Council (reference MR/N028511/1).

show abstract

Caution! Warnings as a Useless Countermeasure to Reduce Overconfidence? An Experimental Evaluation in Light of Enhanced and Dynamic Warning Designs

Cited by 9 publications

References 40 publications

“I Thought I Did Much Better”—Overconfidence in University Exams

“I Thought I Did Much Better”—Overconfidence in University Exams

Gamification and Hazard Communication in Virtual Reality: A Qualitative Study

Developing a reference protocol for structured expert elicitation in health-care decision-making: a mixed-methods study

Contact Info

Product

Resources

About