Do additional features help or hurt category learning? The curse of dimensionality in human learners

Vong, Wai Keen; Hendrickson, Andrew; Navarro, Daniel J.; Perfors, Amy

doi:10.31234/osf.io/bjh68

Cited by 2 publications

(3 citation statements)

References 34 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…While it is plausible that decision-makers encode all relevant stimulus information from the low-dimensional stimuli typically considered in the laboratory, 1 in high-dimensional environments, encoding all available sensory information is inefficient, and can impair learning. This reflects a fundamental computational constraint (known as the curse of dimensionality), which affects both machine-learning algorithms (Hastie, Tibshirani, & Friedman, 2009;Li et al, 2017) and human decision-makers (e.g., Bulgarella & Archer, 1962;Edgell et al, 1996;Pishkin, Bourne, & Fishkin, 1974;Vong et al, 2018).…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Bidirectional influences of information sampling and concept learning.

Braunlich¹,

Love²

2022

Psychological Review

View full text Add to dashboard Cite

Contemporary models of categorization typically tend to sidestep the problem of how information is initially encoded during decision-making. Instead, a focus of this work has been to investigate how, through selective attention, stimulus representations are contorted such that behaviourally-relevant dimensions are accentuated (or "stretched"), and representations of irrelevant dimensions are ignored (or "compressed"). In high-dimensional real-world environments, it is computationally infeasible to sample all available information, and human decision-makers selectively sample information from sources expected to provide relevant information. To address these and other shortcomings, we develop an active sampling model, Sampling Emergent Attention (SEA), which sequentially and strategically samples information sources until the expected cost of information exceeds the expected benefit. The model specifies the interplay of two components, one involved in determining the expected utility of different information sources and the other in representing knowledge and beliefs about the environment. These two components interact such that knowledge of the world guides information sampling, and what is sampled updates knowledge. Like human decision-makers, the model displays strategic sampling behaviour, such as terminating information search when sufficient information has been sampled and adaptively adjusting the search path in response to previously sampled information. The model also shows human-like failure modes. For example, when information exploitation is prioritized over exploration, the bidirectional influences between information-sampling and learning can lead to the development of beliefs that systematically differ from reality.

show abstract

Section: Introductionmentioning

confidence: 99%

“…In the category-learning literature, stimuli with two-four features are common (e.g.,Nosofsky, 1986;Shepard et al, 1961), and stimuli with 16 features are considered to be high-dimensional (e.g.,Vong, Hendrickson, Navarro, & Perfors, 2018).…”

mentioning

confidence: 99%

Bidirectional influences of information sampling and concept learning.

Braunlich¹,

Love²

2022

Psychological Review

View full text Add to dashboard Cite

show abstract

“…Overfitting and, even more, brute force memorization should exclude generalization by definition, even as concerns human beings. For instance, the concepts of capacity ( [24], [25], [26], [27], [28], [29], bias ( [30], [31]), overfitting ( [32], [33]), and generalization ( [34], [35]) have been widely explored in cognitive psychology as well.…”

mentioning

confidence: 99%

Deep Neural Networks as the Semi-classical Limit of Quantum Neural Networks

Marciano,

Chen,

Fabrocini*

et al. 2020

Preprint

View full text Add to dashboard Cite

Our work intends to show that: (1) Quantum Neural Networks (QNN) can be mapped onto spinnetworks, with the consequence that the level of analysis of their operation can be carried out on the side of Topological Quantum Field Theories (TQFT); (2) Deep Neural Networks (DNN) are a subcase of QNN, in the sense that they emerge as the semiclassical limit of QNN; (3) a number of Machine Learning (ML) key-concepts can be rephrased by using the terminology of TQFT. Our framework provides as well a working hypothesis for understanding the generalization behavior of DNN, relating it to the topological features of the graphs structures involved. I. INTRODUCTIONA paradoxical result from [1] according to which DNN memorize the training samples by brute force leaves unexplained where the generalization capabilities of DNN come from. This "apparent" paradox, as it has been dubbed by [2], has led to active discussions by many scholars such as [15]. In any case, in our vision, the overall discussion has empirically proved how far the ML community is from building a principled model of DNN and, therefore, understanding their generalization capabilities.So far, ML techniques have been deployed in order to enhance quantum tasks ([16], [17], [18]), while Quantum Computing (QC) has been used to speed up traditional ML algorithms ([19], [20], [21], [22]). We will follow here a different perspective encompassing DNN as a sub-case (semi-classical limit) of QNN. This pathway has been suggested by the analogy with physics. A topical experiment at the base of the quantum revolution around the beginnings of the 20th century pointed out the existence of the photoelectric effect. As it is notorious, the effect has been explained by Albert Einstein resorting to a corpuscular description of the electromagnetic field, namely to the concept of photons as carriers of "quanta" of light. But, actually, the interpretation of this very seminal experiment clashed with a common perspective on quantum physics, widely spread nowadays even in the physicist's community, and that relies on the naive assumption that quantum means microscopic and classical macroscopic. A rather different pathway consists in moving from a quantum theory, with tested semi-classical limit that corresponds to the classical theory, and investigating the varieties of predictions that can then falsify the quantum theory. This approach allows new predictive power and more robust experimental corroboration, and it is the approach we have been following within this paper.

show abstract

Do additional features help or hurt category learning? The curse of dimensionality in human learners

Cited by 2 publications

References 34 publications

Bidirectional influences of information sampling and concept learning.

Bidirectional influences of information sampling and concept learning.

Deep Neural Networks as the Semi-classical Limit of Quantum Neural Networks

Contact Info

Product

Resources

About