Deep Neural Network Models of Object Recognition Exhibit Human-Like Limitations when Performing Visual Search Tasks

Nicholson, David; Prinz, Astrid A.

doi:10.1101/2020.10.26.354258

Cited by 3 publications

(2 citation statements)

References 89 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Avoiding affirming the consequent can also be subserved by the clarification that ANNs "may simply rely on brute-force memorization and interpolation to learn how to generate the appropriate linguistic outputs in light of prior contexts" (Goldstein et al, 2021) -something which does appear to be true in certain contexts (Zhang, Bengio, Hardt, Recht, & Vinyals, 2016). Others solve this by carefully couching their findings as explicitly correlational where applicable, handin-hand with conceptually analysing the capacity under study (e.g., Lindsay & Miller, 2018;Nicholson & Prinz, 2020). Either way, leaving our syllogisms ambiguous -this includes not explaining that it is Q → P and not P → Q -leads to, or at least does not protect from affirming the consequent.…”

Section: Impediments To Inferencementioning

confidence: 99%

On logical inference over brains, behaviour, and artificial neural networks

Guest¹,

Martin²

2021

Preprint

View full text Add to dashboard Cite

In the cognitive, computational, and neuro- sciences, we often reason about what models (viz., formal and/or computational) represent, learn, or "know", as well as what algorithm they instantiate. The putative goal of such reasoning is to generalize claims about the model in question to claims about the mind and brain. This reasoning process typically presents as inference about the representations, processes, or algorithms the human mind and brain instantiate. Such inference is often based on a model's performance on a task, and whether that performance approximates human behaviour or brain activity. The model in question is often an artificial neural network (ANN) model, though the problems we discuss are generalizable to all reasoning over models. Arguments typically take the form "the brain does what the ANN does because the ANN reproduced the pattern seen in brain activity" or "cognition works this way because the ANN learned to approximate task performance." Then, the argument concludes that models achieve this outcome by doing what people do or having the capacities people have. At first blush, this might appear as a form of modus ponens, a valid deductive logical inference rule. However, as we explain in this article, this is not the case, and thus, this form of argument eventually results in affirming the consequent – a logical or inferential fallacy. We discuss what this means broadly for research in cognitive science, neuroscience, and psychology; what it means for models when they lose the ability to mediate between theory and data in a meaningful way; and what this means for the logic, the metatheoretical calculus, our fields deploy in high-level scientific inference.

show abstract

Section: Impediments To Inferencementioning

confidence: 99%

On logical inference over brains, behaviour, and artificial neural networks

Guest¹,

Martin²

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Examples include automatically recognizing a person in selfie-images in social media platforms [108], diagnosing diseases from scanned medical images in health care [2], autonomous driving in self-driving cars [110], facial recognition [39] or fingerprint recognition [17] in security industry, automatically adding item details from product images in retail industry [4], and searching images instead of text in visual search engine [142] [87].…”

Section: Problem Statementmentioning

confidence: 99%

Evolving Deep Neural Networks with Explanations for Image Classification

Wang¹

View full text Add to dashboard Cite

Image classification problems often face the issues of high dimensionality and large variance within the same class. Deep convolutional neural networks are designed to solve the problem by extracting features using convolutional operations. Researchers have developed complex deep convolutional neural networks to achieve the outstanding performance that outperforms humans. However, the complexity of deep convolutional neural networks brings two side effects. First, the more complex the network architecture is, the harder it is to design. Second, it deteriorates the black-box nature of deep convolutional neural networks, which is harder to explain. To tackle the above two issues, neural architecture search and explainable deep learning have emerged as two promising research areas for automatically designing deep convolutional neural networks and providing explanations of the predictions made by deep learning models, respectively. Evolutionary computation based neural architecture search has been employed to automatically design deep convolutional neural networks that outperform those manually designed, but the computational cost is too high. Surrogate-assisted and transfer learning based methods can be utilised to reduce the computational cost. Furthermore, a branch in explainable deep learning called local approximation does not need machine learning expertise to understand the explanation. The target of local approximation is to find interpretable features. Evolutionary computation has successfully applied to many search problems, but it has never been used in finding local approximation. The overall goal of this thesis is to improve the efficiency of evolutionary neural architecture search in image classification tasks and provide explanations in the inference phase. This can mitigate the major issues -- the difficulty of designing deep convolutional neural networks and the lack of explainability, that significantly affected deep learning being widely used in real-world applications. This thesis proposes a surrogate-assisted particle swarm optimisation (an evolutionary computation algorithm) method to efficiently evolve deep convolutional neural networks. A surrogate model is proposed to predict a better solution from a pair of solutions, and a method for sampling a subset of the dataset as a surrogate dataset is proposed to reduce the computational cost to less than 3 GPU-days, but with very competitive classification accuracies across several benchmark datasets. Next, this thesis proposes to use smaller datasets in multiple source domains to evolve deep convolutional neural networks, and then transfer the learned models to the target domain. This has achieved the goal of accelerating the evolutionary neural architecture search process and improving the generalisation performance, which is supported by the experiment results. Furthermore, this thesis proposes a genetic algorithm (an evolutionary computation algorithm) based method to evolve local explanations to explain the predictions of deep convolutional neural networks. By combining the flexible encoding and the proposed fitness evaluation, the proposed method can efficiently evolve meaning interpretable features in the local explanation. It produces competitive explanations, but 10 times faster than Local Interpretable Model-agnostic Explanations (LIME) -- the state-of-the-art method. Lastly, an evolutionary multi-objective approach is explored to evolve local explanations to reduce the human effort of examining the interpretable features in the local explanations. It adds the second objective of minimising the number of interpretable features, which reduces the human effort in checking them. Additionally, it proposes a method to select only two non-dominated solutions from the pareto front to save the labour cost for end users.

show abstract

On Logical Inference over Brains, Behaviour, and Artificial Neural Networks

Guest

Martin

2023

Comput Brain Behav

View full text Add to dashboard Cite

In the cognitive, computational, and neuro-sciences, practitioners often reason about what computational models represent or learn, as well as what algorithm is instantiated. The putative goal of such reasoning is to generalize claims about the model in question, to claims about the mind and brain, and the neurocognitive capacities of those systems. Such inference is often based on a model’s performance on a task, and whether that performance approximates human behavior or brain activity. Here we demonstrate how such argumentation problematizes the relationship between models and their targets; we place emphasis on artificial neural networks (ANNs), though any theory-brain relationship that falls into the same schema of reasoning is at risk. In this paper, we model inferences from ANNs to brains and back within a formal framework — metatheoretical calculus — in order to initiate a dialogue on both how models are broadly understood and used, and on how to best formally characterize them and their functions. To these ends, we express claims from the published record about models’ successes and failures in first-order logic. Our proposed formalization describes the decision-making processes enacted by scientists to adjudicate over theories. We demonstrate that formalizing the argumentation in the literature can uncover potential deep issues about how theory is related to phenomena. We discuss what this means broadly for research in cognitive science, neuroscience, and psychology; what it means for models when they lose the ability to mediate between theory and data in a meaningful way; and what this means for the metatheoretical calculus our fields deploy when performing high-level scientific inference.

show abstract

Deep Neural Network Models of Object Recognition Exhibit Human-Like Limitations when Performing Visual Search Tasks

Cited by 3 publications

References 89 publications

On logical inference over brains, behaviour, and artificial neural networks

On logical inference over brains, behaviour, and artificial neural networks

Evolving Deep Neural Networks with Explanations for Image Classification

On Logical Inference over Brains, Behaviour, and Artificial Neural Networks

Contact Info

Product

Resources

About