Subjective sampling distributions and conservatism

Wheeler, Gloria E.; Beach, Lee Roy

doi:10.1016/0030-5073(68)90025-1

Cited by 54 publications

(18 citation statements)

References 7 publications

(3 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The findings caution against assuming Bayesian belief formation in models of statistical reasoning about explicit prospective beliefs based on repeated events with known probabilities.Goodman & Tenenbaum, 2015;Vul, 2010) and should be simple for a Bayesian decision-maker.Across elicitation methods and decision scenarios, both novices and experts estimate a biased "wizard-hat" shaped subjective binomial distribution. They over-estimated the tails and underestimated the shoulders of the distribution relative to the "gendarme-hat" shaped (Edgeworth's characterization, per Stigler 1999) normative distribution.The bias in estimates is broadly consistent with prior findings documenting errors people make in generating "sampling distributions" (Kahneman & Tversky, 1972;Peterson, DuCharme and Edwards 1968;Wheeler and Beach 1968). However, unlike prior findings, these results cannot be explained by high error variance and regression to the mean, heterogeneity, misunderstanding of general distribution characteristics, or elicitation-specific biases.…”

supporting

confidence: 84%

Probability Distributions of Common Repeated Events Are Misestimated.

Urminsky

2019

Preprint

View full text Add to dashboard Cite

Accurately estimating the prospective probability distribution arising from repeated events with known probabilities, such as the number of heads in ten coin flips, represents a simple aptitude necessary for explicit Bayesian updating and useful in optimal decisions in the face of future uncertainty. Across elicitation methods and decision scenarios, people express beliefs that are systematically biased relative to the actual distribution. Participant beliefs reflect a "wizard-hat" shaped distribution, over-estimating the tails and under-estimating the shoulders of the distribution, relative to the actual bell-curve shape. While experts are relatively more accurate than novices, both show significant bias. The bias is not explained by regression to the mean, random error or participant heterogeneity, and is exacerbated by increasing the number of repeated events. The findings caution against assuming Bayesian belief formation in models of statistical reasoning about explicit prospective beliefs based on repeated events with known probabilities.Goodman & Tenenbaum, 2015;Vul, 2010) and should be simple for a Bayesian decision-maker.Across elicitation methods and decision scenarios, both novices and experts estimate a biased "wizard-hat" shaped subjective binomial distribution. They over-estimated the tails and underestimated the shoulders of the distribution relative to the "gendarme-hat" shaped (Edgeworth's characterization, per Stigler 1999) normative distribution.The bias in estimates is broadly consistent with prior findings documenting errors people make in generating "sampling distributions" (Kahneman & Tversky, 1972;Peterson, DuCharme and Edwards 1968;Wheeler and Beach 1968). However, unlike prior findings, these results cannot be explained by high error variance and regression to the mean, heterogeneity, misunderstanding of general distribution characteristics, or elicitation-specific biases. The findings challenge the emerging view that the human brain is adept at optimal statistical processing for sufficiently simple tasks, with broad implications for people's ability to accurately assess and manage risk. 2.Study 1 MethodAdult participants were recruited from Amazon MTurk to complete an online survey. A target of 900 participants were requested, yielding 867 completed surveys. Records with duplicate IP addresses, or who failed a basic attention check were removed prior to analysis, yielding 821 valid completes. The participants constituted a novice population, with less than half holding a Bachelor's degree and only 5% identifying themselves as knowledgeable in statistics.Participants read a hypothetical scenario, either about one of three equal-probability events which would be repeated for 10 independent trials (coin-flips, survey sampling or soccer kicks), or about the distribution of height (a control task that could be estimated based on memory rather than probabilistic inferences). To test the accuracy of beliefs about the full probability distribution Participants were randomly assigned to one of t...

show abstract

supporting

confidence: 84%

Probability Distributions of Common Repeated Events Are Misestimated.

Urminsky

2019

Preprint

View full text Add to dashboard Cite

show abstract

“…The biased sampling distribution hypothesis was supported by the observation that subjective sampling distributions were indeed flatter than the objective ones, and substituting these beliefs into Bayes' rule accorded well with reported posterior beliefs (Peterson, DuCharme, & Edwards, 1968;Wheeler & Beach, 1968). On the other hand, a critical weakness of this hypothesis is that it cannot explain the existence of under-reaction with a sample size of 1, which would require that subjects disbelieve the experimenter when they are explicitly told the sampling distribution (i.e., the proportion of red chips in the bag).…”

Section: Under-reaction To Probabilistic Informationmentioning

confidence: 78%

A theory of learning to infer

Dasgupta

Schulz

Tenenbaum

et al. 2019

Preprint

View full text Add to dashboard Cite

Bayesian theories of cognition assume that people can integrate probabilities rationally.However, several empirical findings contradict this proposition: human probabilistic inferences are prone to systematic deviations from optimality. Puzzlingly, these deviations sometimes go in opposite directions. Whereas some studies suggest that people under-react to prior probabilities (base rate neglect), other studies find that people under-react to the likelihood of the data (conservatism). We argue that these deviations arise because the human brain does not rely solely on a general-purpose mechanism for approximating Bayesian inference that is invariant across queries. Instead, the brain is equipped with a recognition model that maps queries to probability distributions. The parameters of this recognition model are optimized to get the output as close as possible, on average, to the true posterior. Because of our limited computational resources, the recognition model will allocate its resources so as to be more accurate for high probability queries than for low probability queries. By adapting to the query distribution, the recognition model "learns to infer." We show that this theory can explain why and when people under-react to the data or the prior, and a new experiment demonstrates that these two forms of under-reaction can be systematically controlled by manipulating the query distribution. The theory also explains a range of related phenomena: memory effects, belief bias, and the structure of response variability in probabilistic reasoning. We also discuss how the theory can be integrated with prior sampling-based accounts of approximate inference.

show abstract

“…Since, as found in psychological literature, conservatism is also due to the subjects' misperception of the data [16,17,18], the effects found in our comparison can also be due to an incorrect weighting given to the ambiguous data --subjects assume ambiguous data occurs a fraction of time which is not equal to the prescribed 0.2 . Models will be developed which take into account the above mentioned cognitive biases.…”

Section: A Nornuiye Moandkandr Subor-mentioning

confidence: 87%

“…Most of these studies indicate that the subjective estimates of event probabilities are monotonically related to those specified by the Bayes' rule, but are conservative. Conservatism has been attributed to the inability of the subjects to integrate the meaning of the data into a probability estimate and their lack of understanding of the implications of the data, i.e., usage of inaccurate probability distributions [16,17,18]. The subjects' estimates were based on a different data-generating model than the one used by the experimenters [17].…”

mentioning

confidence: 99%

“…The subjects' estimates were based on a different data-generating model than the one used by the experimenters [17]. Wheeler and Beach [18] found that the subjective probability estimates became less conservative after the subjects were trained about the probability distributions. Peterson et al [16] conclude that the sampling distributions used by subjects became more conservative, as the complexity of the task was increased.…”

mentioning

confidence: 99%

See 1 more Smart Citation

A model of distributed team information processing under ambiguity

Mallubhatla

Pattipati

Kleinman

et al.

Conference Proceedings., IEEE International Conference on Systems, Man and Cybernetics

View full text Add to dashboard Cite

This paper considers distributed information processing by a three person team operating in a binary hypothesis testing environment. The team is hierarchical with a primary decisionmaker @MO) and two subordinate DMs (DMl and DM2). Given a set of measurements, the team has to decide whether a contact is a "threat" or a "neutral". The subordinates are experts, one at detecting threats and the other at detecting neutrals. The team has access to noisy measurements from three sensors; one global, shared by all three DMs, and two local, dedicated to each of the two subordinates.In addition to being noisy, the measurements from the sensors of the two subordinates are ambiguous at times, i.e., they provide no information pertaining to the task. Based on their own private and global data, the subordinates transmit their opinions on the task type, along with confidence levels of their opinions, to the primary decisionmaker. The primary DM makes the final team decision based on the reports of the subordinates and the global measurement, and attaches a confidence level to the decision. A normative model for the distributed detection process is developed. The normative predictions are compared with the experimental data to identify cognitive biases of human DMs. A normative-descriptive model, which accounts for these biases, is developed, and has been shown to provide excellent match with the experimental data.. . o t r v a mThe work presented in this paper was motivated by distributed decision problems in a naval battle group. Several of the present day complex military command and control problems require information processing and decisionmaking by a geographically distributed human team, usually structured hierarchically with a commander and several subordinates. The human team must coordinate their information, resources and activities in order to attain common goals in what is generally a dynamic and uncertain environment. It is not very uncommon in such an environment that the sensors (radar, sonar, etc.) of the DMs breakdown once-in a while. Whenever a sensor breakdown occurs, the information provided by the sensor is completely ambiguous, Le, it provides no information pertaining to the task In this paper, we are concerned with the problem of modeling information processing by a three-person hierarchical team to identify the target type (a friend or a foe) based on noisy, ambiguous and distributed measurement data. Our modeling approach is normativedescriptive in that we employ optimal (normative) solution of the distributed hypothesis testing problem as a baseline, but mocllfy this solution by placing psychologically interpretable (descriptive) constraints and structure on the decisionmaker's perceptual and cognitive processes. The resulting normative-descriptive model thus has the predictive capabilities needed for system design, as well as the ability to provide realistic predictions of human team performance. B. B&gu&The research presented in this paper is an extension of our previous work on modeling human teams...

show abstract

Subjective sampling distributions and conservatism

Cited by 54 publications

References 7 publications

Probability Distributions of Common Repeated Events Are Misestimated.

Probability Distributions of Common Repeated Events Are Misestimated.

A theory of learning to infer

A model of distributed team information processing under ambiguity

Contact Info

Product

Resources

About