Sample proportions and subjective probability revisions

Beach, Lee Roy; Wise, James A.; Barclay, Scott

doi:10.1016/0030-5073(70)90014-0

Cited by 33 publications

(18 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Subjects observed a process of coloured chips being drawn from a bag and on each trial estimated the probability that the bag was predominantly a particular colour. The results were relatively simple to replicate, but the artificial nature of the experiment was a cause for criticism (Beach et al. , 1970).…”

Section: Internal Consistency or Coherencementioning

confidence: 99%

See 1 more Smart Citation

The ‘Heuristics and Biases’ Bias in Expert Elicitation

Kynn

2007

Journal of the Royal Statistical Society Series A: Statistics in Society

173

119

View full text Add to dashboard Cite

In the early 1970s Tversky and Kahneman published a series of papers on 'heuristics and biases' describing human inadequacies in assessing probabilities, culminating in a highly popular article in Science. This seminal research has been heavily cited in many fields, including statistics, as the definitive research on probability assessment. Curiously, although this work was debated at the time and more recent work has largely refuted many of the claims, this apparent heuristics and biases bias in elicitation research has gone unremarked. Over a decade of research into the frequency effect, the importance of framing, and cognitive models more generally, has been almost completely ignored by the statistical literature on expert elicitation. To remedy this situation, this review offers a guide to the psychological research on assessing probabilities, both old and new, and gives concrete guidelines for eliciting expert knowledge.

show abstract

Section: Internal Consistency or Coherencementioning

confidence: 99%

“…The second observation is that subjects tended to be more accurate as the complexity of the task increased. One suggestion is that it is difficult to control for distracting features in the simplistic book bag and poker chip experiments (Manz, 1970; Beach et al. , 1970).…”

Section: Internal Consistency or Coherencementioning

confidence: 99%

The ‘Heuristics and Biases’ Bias in Expert Elicitation

Kynn

2007

Journal of the Royal Statistical Society Series A: Statistics in Society

173

119

View full text Add to dashboard Cite

show abstract

“…updating tasks when sequentially processing data (Beach, Wise and Barclay 1970;Lopes 1985, 1987, Hogarth and Einhorn 1992. For example, Lopes (1985) shows that when signals are presented sequentially then subjects tend to average across observations to calculate their subjective posterior probability instead of updating the already known information according to Bayes rule.…”

Section: Consideration Of Weight and Strength Given One Set Of Segregmentioning

confidence: 99%

How Do People Take into Account Weight, Strength and Quality of Segregated vs. Aggregated Data? Experimental Evidence

Kraemer

Weber

2004

Journal of Risk and Uncertainty

View full text Add to dashboard Cite

In this experimental study we investigated how people aggregate two sets of signals about the state of the world to reach a single probability judgment. The signal sets may differ in the way signals are presented, in their number as well as their quality. By varying the presentation mode of the signals we investigated how people deal with segregated and aggregated evidence. We investigated whether subjects sufficiently take into account weight (number of signals), strength (composition) and quality of the information provided. The results indicate that consideration of the weight and strength of signals strongly depends on the type of their presentation. Particular patterns can be identified which determine if weight and/or strength are either under-or overweighted. JEL C25, C91, D8Keywords: weight and strength of information, Bayes' rule, heuristics 1 Imagine you are in a situation where you want to update your initial beliefs based upon a variety of information sources. For instance, one such situation would be if you wanted to assess the probability of the stock market going up the next trading day given such information as research reports, market prices and colleagues' opinions. The question is how to evaluate the existing evidence and how to combine these multiple pieces of evidence coming from different sources in order to reach a single probability judgment.In many situations, it is possible to distinguish a set of available evidence by the dimensions of extremeness (strength), credibility (weight) and quality of the pieces it consists of. For example, strength can refer to the proportion of colleagues who think that the market will go up the next day; weight can refer to the total number of opinions and quality can refer to the knowledge of a specific colleague. Hence, strength expresses how representative the evidence is of a specific hypothesis, whereas weight expresses its statistical reliability and quality expresses the reliability of a single observation.Another important distinction is whether evidence represents already aggregated information or whether evidence just consists of multiple pieces of information which still have to be aggregated. Market prices, for example, can be viewed as the aggregated opinions of all market participants; a set of opinions from one's colleagues, however, still has to be aggregated to reach a judgment.All these characteristics, i.e. strength, weight and quality as well as whether evidence is already aggregated or not, affect posterior beliefs which are determined based on all available sources of information. For instance, a posteriori beliefs should not only be based on the fact that all colleagues think that the market will go up but it is also important to incorporate the aspect of how many colleagues express this opinion, as larger samples allow for more reliable inference.Thus, it certainly makes a difference if just one or two colleagues express this opinion or the 2 entire research department does. Furthermore, it is also important to consider how ...

show abstract

“…Early evidence linking averaging and conservatism came from experiments showing that subjects' ratings in Bayesian tasks were often more like estimates of population proportion than like inferences from Bayes's rule (Beach, Wise , & Barclay, 1970;Shanteau, 1970Shanteau, , 1972. Shanteau hypothesized that the data could be modeled by a rule in which the response at each serial position is given by a weighted average of the scale values of the previous and current sample events .…”

Section: Averaging and Conservatismmentioning

confidence: 99%

Averaging rules and adjustment processes in Bayesian inference

Lopes

1985

Bull. Psychon. Soc.

View full text Add to dashboard Cite

Two empirically well-supported research findings in the judgment literature are (1)that human judgments often appear to follow an averaging rule, and (2) that judgments in Bayesian inference tasks are usually conservative relative to optimal judgments. This paper argues that both averaging and conservatism in the Bayesian task occur because subjects produce their judgments by using an adjustment strategy that is qualitatively equivalent to averaging. Two experiments are presented that show qualitative errors in the direction of revisions in the Bayesian task that are well accounted for by the simple adjustment strategy. Also noted is the tendency for subjects in one experiment to evaluate sample evidence according to representativeness rather than according to relative likelihood. The final discussion describes task variables that predispose subjects toward averaging processes.The Bayesian inference task is usually instantiated in terms of the " bookbag and poker chips paradigm" in which there are two competing hypotheses involving populations of binary events. For example, there may be two bookbags, one containing 70 red poker chips and 30 blue poker chips (the " red bag") and another containing 30 red chips and 70 blue chips (the " blue bag"). In most experiments, the experimenter ostensibly selects a bag at random and then draws samples from it one or more times. These samples are shown to the subject, usually sequentially, and the subject rates the strength of his or her belief about which bookbag was sampled.

show abstract

Sample proportions and subjective probability revisions

Cited by 33 publications

References 6 publications

The ‘Heuristics and Biases’ Bias in Expert Elicitation

The ‘Heuristics and Biases’ Bias in Expert Elicitation

How Do People Take into Account Weight, Strength and Quality of Segregated vs. Aggregated Data? Experimental Evidence

Averaging rules and adjustment processes in Bayesian inference

Contact Info

Product

Resources

About