Are you sure about that? On the origins of confidence in concept learning

Stojić, Hrvoje; Eldar, Eran; Bassam, Hassan; Dayan, Peter; Dolan, Raymond J.

doi:10.32470/ccn.2018.1197-0

Cited by 3 publications

(3 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In future work, we aim to extend our model to assess the neural correlates of generalization and uncertainty-guided exploration, test how people track their uncertainty in other domains that require functional knowledge (cf. Stojic, Eldar, Bassam, Dayan, & Dolan, 2018), as well as extend our model further to real world decision making such as consumer behavior.…”

Section: Discussionmentioning

confidence: 98%

It’s new, but is it good? How generalization and uncertainty guide the exploration of novel options.

Stojić¹,

Schulz²,

Analytis³

et al. 2020

Journal of Experimental Psychology: General

View full text Add to dashboard Cite

How do people decide whether to try out novel options as opposed to tried-and-tested ones? We argue that they infer a novel option's reward from contextual information learned from functional relations and take uncertainty into account when making a decision. We propose a Bayesian optimization model to describe their learning and decision making. This model relies on similarity-based learning of functional relationships between features and rewards, and a choice rule that balances exploration and exploitation by combining predicted rewards and the uncertainty of these predictions. Our model makes two main predictions. First, decision makers who learn functional relationships will generalize based on the learned reward function, choosing novel options only if their predicted reward is high. Second, they will take uncertainty about the function into account, and prefer novel options that can reduce this uncertainty. We test these predictions in three preregistered experiments in which we examine participants' preferences for novel options using a feature-based multi-armed bandit task in which rewards are a noisy function of observable features. Our results reveal strong evidence for functional exploration and moderate evidence for uncertainty-guided exploration. However, whether or not participants chose a novel option also depended on their attention, as well as reflecting on the value of the options. These results advance our understanding of people's reactions in the face of novelty.

show abstract

Section: Discussionmentioning

confidence: 98%

It’s new, but is it good? How generalization and uncertainty guide the exploration of novel options.

Stojić¹,

Schulz²,

Analytis³

et al. 2020

Journal of Experimental Psychology: General

View full text Add to dashboard Cite

show abstract

“…What if the model has to extrapolate outside the convex hull of its training data? Studies in psychology and cognitive science show that humans and machines are usually less confident in their decisions and predictions when they extrapolate instead of interpolating, and a correlation exists between human confidence and the correctness of predictions [43].…”

Section: Studies On Promoting Accountability and Transparencymentioning

confidence: 99%

“…Let us be clear: we do not imply that all samples outside the convex hull of the training set should be considered out-of-distribution. Extrapolation and learning frequently supplement each other, but evidence suggests that both humans and AI models are more susceptible to mistakes when they extrapolate as opposed to when they interpolate [9,39,43,50]. This is sometimes reported as extrapolation tasks being more difficult to perform correctly [15,21,29].…”

Section: Studies On Extrapolation and Geometry Of Datasetsmentioning

confidence: 99%

To what extent should we trust AI models when they extrapolate?

Yousefzadeh¹,

Cao²

2022

Preprint

View full text Add to dashboard Cite

Many applications affecting human lives rely on models that have come to be known under the umbrella of machine learning and artificial intelligence. These AI models are usually complicated mathematical functions that make decisions and predictions by mapping from an input space to an output space. Stakeholders are interested to know the rationales behind models' decisions; that understanding requires knowledge about models' functional behavior. We study this functional behavior in relation to the data used to create the models. On this topic, scholars have often assumed that models do not extrapolate, i.e., they learn from their training samples and process new input by interpolation. This assumption is questionable: we show that models extrapolate frequently; the extent of extrapolation varies and can be socially consequential. We demonstrate that extrapolation happens for a substantial portion of datasets more than one would consider reasonable. How can we trust models if we do not know whether they are extrapolating? Given a model trained to recommend clinical procedures for patients, can we trust the recommendation when the model considers a patient older or younger than all the samples in the training set? If the training set is mostly Whites, how do we measure the extent to which we can trust its recommendations about Black and Hispanic patients? How do we know if extrapolation is significant for a given sample if we do not compare the sample to the training data? Which dimension (race, gender, or age) does extrapolation happen? Even if a model is trained on people of all races, it still may extrapolate in significant ways related to race. So the leading question is, to what extent can we trust AI models when they process inputs that fall outside their training set? This paper investigates several social applications of AI, showing how models extrapolate without notice. The difficulty of auditing is that datasets have many features, thus hard to review individually. To solve this problem, we use a systematic method of geometric analysis. We also look at different sub-spaces of extrapolation for specific individuals subject to AI models and report how these extrapolations can be interpreted, not mathematically, but from a humanistic point of view. We recommend that AI pipelines include this auditing approach. If extrapolation is statistically excessive, the model's output should be sent for review by human experts and provide information about the extent and dimensions of extrapolation.CCS Concepts: • Computing methodologies → Machine learning; • Social and professional topics → Governmental regulations; • Theory of computation → Randomness, geometry and discrete structures.

show abstract

Metacognitive resources for adaptive learning⋆

Cortese

2022

Neuroscience Research

View full text Add to dashboard Cite

Are you sure about that? On the origins of confidence in concept learning

Cited by 3 publications

References 13 publications

It’s new, but is it good? How generalization and uncertainty guide the exploration of novel options.

It’s new, but is it good? How generalization and uncertainty guide the exploration of novel options.

To what extent should we trust AI models when they extrapolate?

Metacognitive resources for adaptive learning⋆

Contact Info

Product

Resources

About