Familiarity‐Matching: An Ecologically Rational Heuristic for the Relationships‐Comparison Task

Shirasuna, Masaru; Honda, Hidehito; Matsuka, Toshihiko; Ueda, Kazuhiro

doi:10.1111/cogs.12806

Cited by 3 publications

(7 citation statements)

References 66 publications

(122 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…): This task has recently been proposed, also to examine human inferences in the context of heuristics (specifically, the adaptive use of heuristics) [43][44][45]. In this task, the task structure was that objects were presented not only in alternatives but also in a question sentence format and more directly asked participants' general knowledge, such as "Which country is City Q in, Country A or Country B?"…”

Section: Relationships Comparison Task (Fig 2 Lowermentioning

confidence: 99%

“…In the present study, we used a relationships comparison task as another task to confirm the consistency and validity of our experimental results. Regarding the objects presented to participants, we selected 25 city names based on the rates of correct judgments in 13 the original study [44]. For the process of selecting the questions and the question list, see Supporting Information 2.…”

Section: Relationships Comparison Task (Fig 2 Lowermentioning

confidence: 99%

See 1 more Smart Citation

Can individual subjective confidence in prior questions predict group performance in future questions?

Shirasuna¹,

Honda²

2022

Preprint

Self Cite

View full text Add to dashboard Cite

When people have to solve many tasks, they can aggregate diverse individuals’ judgments using the majority rule, which often improves the accuracy of judgments (wisdom of crowds). When aggregating individuals’ judgments, people sometimes decide whose judgments they should accept based on their subjective confidence. However, can confidence in one task set predict performance not only in the same (prior) task set, but also in another (future) task set? We examined this issue through computer simulations using behavioral data obtained from binary-choice experimental tasks. In our simulations, we developed a “training-test” approach: we split the questions used in behavioral experiments into “training questions” (as prior questions) and “test questions” (as future questions), similar to the cross validation method in machine learning. We found that (i) through analyses of behavioral data, confidence in one question could predict accuracy for the same question, but not always well in another question. (ii) Through a computer simulation of the accordance of two individuals’ judgments, individuals rated with high confidence in one training question tended to make less diverse judgments between each other in test questions. (iii) Through a computer simulation of group judgments, the accuracy of groups constructed by individuals with high confidence in the training question(s) sometimes largely decreased in the test questions, especially when only one training question was available. These results suggest that if people aggregate diverse individuals in terms of subjective confidence in prior questions, they will likely avoid decreasing performance in future questions. We believe that our simulations following a “training-test” approach provide practical implications for keeping groups’ ability to solve future tasks and emphasize the importance of a diversity of confidence in a group.

show abstract

Section: Relationships Comparison Task (Fig 2 Lowermentioning

confidence: 99%

Section: Relationships Comparison Task (Fig 2 Lowermentioning

confidence: 99%

Can individual subjective confidence in prior questions predict group performance in future questions?

Shirasuna¹,

Honda²

2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…To obtain the first evidence about the relationships between individuals' subjective confidence and group judgments' accuracy, the present study conducted behavioral experiments and computer simulations. To investigate the validity of the results, two types of inferential tasks were used: a population inference task [12][13][14][15][16] and a relationships comparison task [17][18][19] (both of which are described later in detail). First, online behavioral experiments were conducted wherein individuals were asked to answer binary choice questions and rate subjective confidence about their judgments.…”

Section: Study Outlinementioning

confidence: 99%

“…Relationships comparison task (Fig. 2, lower): A relationships comparison task is a task recently proposed for examining human inferences (especially, the adaptive use of heuristics within the adaptive toolbox framework) [18], [19] . In this task, the task structure is that objects are presented not only in alternatives but also in a question sentence format, for example "Which country is city Q in, country A or country B?"…”

Section: Tasks and Materialsmentioning

confidence: 99%

“…In the present study, the relationships comparison task was used as another task to investigate the effects of individuals' confidence in their judgments on group judgments. Regarding the objects that were presented to participants, we selected 50 cities and 21 countries used in Study 2b of the previous study [18] based on the correct rate of each question in the original study. For details on the process of selecting questions and the question list, see Supplementary Information 2.…”

Section: Tasks and Materialsmentioning

confidence: 99%

See 1 more Smart Citation

Do low-confidence individuals decrease group judgments’ accuracy? Investigations in terms of the wisdom of crowds framework

Shirasuna¹,

Honda²

2021

Preprint

Self Cite

View full text Add to dashboard Cite

In group judgments in a binary choice task, the judgments of individuals with low confidence (i.e., they feel that the judgment was not correct) may be regarded as unreliable. Previous studies have shown that aggregating individuals’ diverse judgments can lead to high accuracy in group judgments, a phenomenon known as the wisdom of crowds. Therefore, if low-confidence individuals make diverse judgments between individuals and the mean of accuracy of their judgments is above the chance level (.50), it is likely that they will not always decrease the accuracy of group judgments. To investigate this issue, the present study conducted behavioral experiments using binary choice inferential tasks, and computer simulations of group judgments by manipulating group sizes and individuals’ confidence levels. Results revealed that (I) judgment patterns were highly similar between individuals regardless of their confidence levels; (II) the low-confidence group could make judgments as accurate as the high-confidence group, as the group size increased; and (III) even if there were low-confidence individuals in a group, they generally did not inhibit group judgment accuracy. The results suggest the usefulness of low-confidence individuals’ judgments in a group and provide practical implications for real-world group judgments.

show abstract

Can individual subjective confidence in training questions predict group performance in test questions?

Shirasuna

Honda

2023

PLoS ONE

View full text Add to dashboard Cite

When people have to solve many tasks, they can aggregate diverse individuals’ judgments using the majority rule, which often improves the accuracy of judgments (wisdom of crowds). When aggregating judgments, individuals’ subjective confidence is a useful cue for deciding which judgments to accept. However, can confidence in one task set predict performance not only in the same task set, but also in another? We examined this issue through computer simulations using behavioral data obtained from binary-choice experimental tasks. In our simulations, we developed a “training-test” approach: We split the questions used in the behavioral experiments into “training questions” (as questions to identify individuals’ confidence levels) and “test questions” (as questions to be solved), similar to the cross-validation method in machine learning. We found that (i) through analyses of behavioral data, confidence in a certain question could predict accuracy in the same question, but not always well in another question. (ii) Through a computer simulation for the accordance of two individuals’ judgments, individuals with high confidence in one training question tended to make less diverse judgments in other test questions. (iii) Through a computer simulation of group judgments, the groups constructed from individuals with high confidence in the training question(s) generally performed well; however, their performance sometimes largely decreased in the test questions especially when only one training question was available. These results suggest that when situations are highly uncertain, an effective strategy is to aggregate various individuals regardless of confidence levels in the training questions to avoid decreasing the group accuracy in test questions. We believe that our simulations, which follow a “training-test” approach, provide practical implications in terms of retaining groups’ ability to solve many tasks.

show abstract

Familiarity‐Matching: An Ecologically Rational Heuristic for the Relationships‐Comparison Task

Cited by 3 publications

References 66 publications

Can individual subjective confidence in prior questions predict group performance in future questions?

Can individual subjective confidence in prior questions predict group performance in future questions?

Do low-confidence individuals decrease group judgments’ accuracy? Investigations in terms of the wisdom of crowds framework

Can individual subjective confidence in training questions predict group performance in test questions?

Contact Info

Product

Resources

About