Deep Networks Can Resemble Human Feed-forward Vision in Invariant Object Recognition

Kheradpisheh, Saeed Reza; Ghodrati, Masoud; Ganjtabesh, Mohammad; Masquelier, Timothée

doi:10.1038/srep32672

Cited by 164 publications

(123 citation statements)

References 89 publications

Supporting

Mentioning

113

Contrasting

Order By: Relevance

“…Interestingly, further developments of these early computational models have led to modern deep convolutional neural networks (DCNNs), which have powered recent breakthroughs in computer vision 23 as well as many other domains. Although these network models are not constrained by experimental data, they have nonetheless been shown to provide an even better fit than earlier models to both behavioral 18,24,25 and electrophysiological 26,27 data (but see Ref. 28).…”

Section: Introductionmentioning

confidence: 99%

Beyond the feedforward sweep: feedback computations in the visual cortex

Kreiman

Serre

2020

Annals of the New York Academy of Sciences

View full text Add to dashboard Cite

Visual perception involves the rapid formation of a coarse image representation at the onset of visual processing, which is iteratively refined by late computational processes. These early versus late time windows approximately map onto feedforward and feedback processes, respectively. State‐of‐the‐art convolutional neural networks, the main engine behind recent machine vision successes, are feedforward architectures. Their successes and limitations provide critical information regarding which visual tasks can be solved by purely feedforward processes and which require feedback mechanisms. We provide an overview of recent work in cognitive neuroscience and machine vision that highlights the possible role of feedback processes for both visual recognition and beyond. We conclude by discussing important open questions for future research.

show abstract

Section: Introductionmentioning

confidence: 99%

Beyond the feedforward sweep: feedback computations in the visual cortex

Kreiman

Serre

2020

Annals of the New York Academy of Sciences

View full text Add to dashboard Cite

show abstract

“…[59,60]) including the CNN setting (e.g. [61,62]), lack the capability of generalisation to novel objects, which is a crucial requirement in prosthetic hand applications. To address this issue, we either need a very large amount of training data or we can capitalise on the flexibility of deep learning system to generalise based on learning abstract representation of different classes of training data.…”

Section: Object Classification Versus Grasp Identificationmentioning

confidence: 99%

Deep learning-based artificial vision for grasp classification in myoelectric hands

et al. 2017

View full text Add to dashboard Cite

Objective. Computer vision-based assistive technology solutions can revolutionise the quality of care for people with sensorimotor disorders. The goal of this work was to enable trans-radial amputees to use a simple, yet efficient, computer vision system to grasp and move common household objects with a two-channel myoelectric prosthetic hand. Approach. We developed a deep learning-based artificial vision system to augment the grasp functionality of a commercial prosthesis. Our main conceptual novelty is that we classify objects with regards to the grasp pattern without explicitly identifying them or measuring their dimensions. A convolutional neural network (CNN) structure was trained with images of over 500 graspable objects. For each object, 72 images, at intervals, were available. Objects were categorised into four grasp classes, namely: pinch, tripod, palmar wrist neutral and palmar wrist pronated. The CNN setting was first tuned and tested offline and then in realtime with objects or object views that were not included in the training set. Main results. The classification accuracy in the offline tests reached for the seen and for the novel objects; reflecting the generalisability of grasp classification. We then implemented the proposed framework in realtime on a standard laptop computer and achieved an overall score of in classifying a set of novel as well as seen but randomly-rotated objects. Finally, the system was tested with two trans-radial amputee volunteers controlling an i-limb UltraTM prosthetic hand and a motion controlTM prosthetic wrist; augmented with a webcam. After training, subjects successfully picked up and moved the target objects with an overall success of up to . In addition, we show that with training, subjects’ performance improved in terms of time required to accomplish a block of 24 trials despite a decreasing level of visual feedback. Significance. The proposed design constitutes a substantial conceptual improvement for the control of multi-functional prosthetic hands. We show for the first time that deep-learning based computer vision systems can enhance the grip functionality of myoelectric hands considerably.

show abstract

“…The proposed DL‐MO framework was based on a fundamental hypothesis that there exists similarity between the state‐of‐the‐art CNN architectures and human visual system in object detection. This hypothesis was at least partially supported by several prior studies, which systematically investigated the correlation between CNN architectures and human neural response using carefully designed psychophysical experiments . Nonetheless, the corresponding degree of similarity varied a lot across different CNN architectures, which could be attributed to the different feature representation power of those CNNs.…”

Section: Discussionmentioning

confidence: 69%

“…experiments. [41][42][43] Nonetheless, the corresponding degree of similarity varied a lot across different CNN architectures, which could be attributed to the different feature representation power of those CNNs. If using a different pretrained CNN architecture in the DL-MO framework, the strength of the correlation and agreement with HOs may be altered.…”

Section: Discussionmentioning

confidence: 99%

A deep learning‐ and partial least square regression‐based model observer for a low‐contrast lesion detection task in CT

Gong

Leng

et al. 2019

Medical Physics

View full text Add to dashboard Cite

Purpose This work aims to develop a new framework of image quality assessment using deep learning-based model observer (DL-MO) and to validate it in a low-contrast lesion detection task that involves CT images with patient anatomical background. Methods The DL-MO was developed using the transfer learning strategy to incorporate a pretrained deep convolutional neural network (CNN), a partial least square regression discriminant analysis (PLS-DA) model and an internal noise component. The CNN was previously trained to achieve the state-of-the-art classification accuracy over a natural image database. The earlier layers of the CNN were used as a deep feature extractor, with the assumption that similarity exists between the CNN and the human visual system. The PLSR model was used to further engineer the deep feature for the lesion detection task in CT images. The internal noise component was incorporated to model the inefficiency and variability of human observer (HO) performance, and to generate the ultimate DL-MO test statistics. Seven abdominal CT exams were retrospectively collected from the same type of CT scanners. To compare DL-MO with HO, 12 experimental conditions with varying lesion size, lesion contrast, radiation dose, and reconstruction types were generated, each condition with 154 trials. CT images of a real liver metastatic lesion were numerically modified to generate lesion models with four lesion sizes (5, 7, 9, and 11 mm) and three contrast levels (15, 20, and 25 HU). The lesions were inserted into patient liver images using a projection-based method. A validated noise insertion tool was used to synthesize CT exams with 50% and 25% of routine radiation dose level. CT images were reconstructed using the weighted filtered back projection algorithm and an iterative reconstruction algorithm. Four medical physicists performed a two-alternative forced choice (2AFC) detection task (with multislice scrolling viewing mode) on patient images across the 12 experimental conditions. DL-MO was operated on the same datasets. Statistical analyses were performed to evaluate the correlation and agreement between DL-MO and HO. Results A statistically significant positive correlation was observed between DL-MO and HO for the 2AFC low-contrast detection task that involves patient liver background. The corresponding Pearson product moment correlation coefficient was 0.986 [95% confidence interval (0.950, 0.996)]. Bland–Altman agreement analysis did not indicate statistically significant differences. Conclusions The proposed DL-MO is highly correlated with HO in a low-contrast detection task that involves realistic patient liver background. This study demonstrated the potential of the proposed DL-MO to assess image quality directly based on patient images in realistic, clinically relevant CT tasks.

show abstract

Deep Networks Can Resemble Human Feed-forward Vision in Invariant Object Recognition

Cited by 164 publications

References 89 publications

Beyond the feedforward sweep: feedback computations in the visual cortex

Beyond the feedforward sweep: feedback computations in the visual cortex

Deep learning-based artificial vision for grasp classification in myoelectric hands

A deep learning‐ and partial least square regression‐based model observer for a low‐contrast lesion detection task in CT

Contact Info

Product

Resources

About