Hypothesis testing as risk behaviour with regard to beliefs

Abstract

In this paper hypothesis‐testing behaviour is compared to risk‐taking behaviour. It is proposed that choosing a suitable test for a given hypothesis requires making a preposterior analysis of two aspects of such a test: the probability of obtaining supporting evidence and the evidential value of this evidence. This consideration resembles the one a gambler makes when choosing among bets, each having a probability of winning and an amount to be won. A confirmatory testing strategy can be defined within this framework as a strategy directed at maximizing either the probability or the value of a confirming outcome. Previous theories on testing behaviour have focused on the human tendency to maximize the probability of a confirming outcome. In this paper, two experiments are presented in which participants tend to maximize the confirming value of the test outcome. Motivational factors enhance this tendency dependent on the context of the testing situation. Both this result and the framework are discussed in relation to other studies in the field of testing behaviour. Copyright © 2000 John Wiley & Sons, Ltd.

Schrödinger's Category: Active Learning in the Face of Label Ambiguity

“…The mere presence of a fuzzy boundary pushed their sampling patterns deep within member space. This pattern of behavior suggests a tendency to avoid uncertainty during learning, a finding that comports with previous work that has shown people avoid ambiguity or risk when making decisions or sampling information outside of the context of category learning (Liefgreen et al, 2020;Poletiek & Berndsen, 2000;Rode et al, 1999). Importantly, the present work offers a novel extension of this previously observed ambiguity avoidance tendency to the category learning realm.…”

Strategies for selecting and evaluating information

“…Although only a minority of participants explicitly stated this in their think-a-loud responses, selecting the query 'primary item stolen' even in conditions in which querying 'burglary time' could have led to the identification of a frontrunner with a higher probability of being the culprit could also be the product of a frontrunner preference mitigated by risk aversion. This finding fits with work by Poletiek and Berndsen (2000), who conceptualised hypothesis-testing behaviour as risk-taking behaviour and illustrated that people displayed certain biases (though in the risk-taking direction) when carrying out a pre-posterior analysis of the probability of obtaining supporting evidence and the evidential value of this evidence. Though not in line with the findings of Poletiek and Berndsen (2000), the 'frontrunner risk aversion' tendency described above echoes the results of Skov and Sherman (1986) who frame this tendency as being a form of confirmatory strategy behaviour (we will explore this more directly in our stepwise information search paradigm -Experiment 4).…”

“…This finding fits with work by Poletiek and Berndsen (2000), who conceptualised hypothesis-testing behaviour as risk-taking behaviour and illustrated that people displayed certain biases (though in the risk-taking direction) when carrying out a pre-posterior analysis of the probability of obtaining supporting evidence and the evidential value of this evidence. Though not in line with the findings of Poletiek and Berndsen (2000), the 'frontrunner risk aversion' tendency described above echoes the results of Skov and Sherman (1986) who frame this tendency as being a form of confirmatory strategy behaviour (we will explore this more directly in our stepwise information search paradigm -Experiment 4). In the present experiment, we found this inclination to be additionally associated with a form of zero-sum reasoning.…”

“…Once again this suggests that participants, when considering the trade-off between the perceived evidential value of outcomes and the perceived probability of obtaining outcomes, trade some evidential value in favour of a query that will allow them to obtain a frontrunner with a higher probability (albeit with a lower chance of this frontrunner being the true culprit). These findings therefore demonstrated that known biases in decision making under uncertainty, such as risk aversion, also play a role in seeking information under uncertainty, at times leading participants to ultimately lose information (Poletiek & Berndsen, 2000).…”

“…Overall, the identification of zero-sum thinking and risk averse behaviour in an information seeking paradigm contributes to the existing literature by bridging the gap between known reasoning fallacies in Bayesian probabilistic reasoning tasks and information-seeking principles, two factors that are rarely considered in conjunction Coenen et al, 2018). To our knowledge only two studies identified risk aversion in information seeking (Poletiek & Berndsen, 2000; Wakabe, Sato, Watamura &Takano, 2012) although one of these was not done within a strictly Bayesian framework (Wakabe et al, 2012). Future work should therefore further investigate the presence of this phenomenon in information seeking paradigms using a purely information-theoretic set-up given it allows one to naturally capture confirmatory strategies both as search preferences aimed to maximise either the probability of a confirming outcome and/or the value of that outcome.…”