Fast Conceptor Classifier in Pre-trained Neural Networks for Visual Recognition

Qian, Guangwu; Zhang, Lei; Zhang, Qianjun

doi:10.1007/978-3-319-59081-3_35

Cited by 4 publications

(3 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Specifically, the ResNet-50 is a 50-layer CNN (48 convolutional layers, one MaxPool layer, and one average pool layer). The ResNet-50 was pretrained on large datasets such as ImageNet Large Scale Visual Recognition Challenge [25] and Microsoft Coco [26] during 90 epochs (passes through the dataset) with a 5-epoch warm-up.…”

Section: Automatic Ai Glaucoma Classifiermentioning

confidence: 99%

Deep Learning Application to Detect Glaucoma with a Mixed Training Approach: Public Database and Expert-Labeled Glaucoma Population

Cellini,

Caamaño,

Carrasco

et al. 2023

Ophthalmic Res

View full text Add to dashboard Cite

Introduction: Artificial intelligence has real potential for early identification of ocular diseases such as glaucoma. An important challenge is the requirement for large databases properly selected, which are not easily obtained. We used a relatively original strategy: a glaucoma recognition algorithm trained with fundus images from public databases and then tested and retrained with a carefully selected patient database. Methods: The study’s supervised deep learning method was an adapted version of the ResNet-50 architecture previously trained from 10,658 optic head images (glaucomatous or non-glaucomatous) from seven public databases. A total of 1,158 new images labeled by experts from 616 patients were added. The images were categorized after clinical examination including visual fields in 304 (26%) control images or those with ocular hypertension and 347 (30%) images with early, 290 (25%) with moderate, and 217 (19%) with advanced glaucoma. The initial algorithm was tested using 30% of the selected glaucoma database and then re-trained with 70% of this database and tested again. Results: The results in the initial sample showed an area under the curve (AUC) of 76% for all images, and 66% for early, 82% for moderate, and 84% for advanced glaucoma. After retraining the algorithm, the respective AUC results were 82%, 72%, 89%, and 91%. Conclusion: Using combined data from public databases and data selected and labeled by experts facilitated improvement of the system’s precision and identified interesting possibilities for obtaining tools for automatic screening of glaucomatous eyes more affordably.

show abstract

Section: Automatic Ai Glaucoma Classifiermentioning

confidence: 99%

Deep Learning Application to Detect Glaucoma with a Mixed Training Approach: Public Database and Expert-Labeled Glaucoma Population

Cellini,

Caamaño,

Carrasco

et al. 2023

Ophthalmic Res

View full text Add to dashboard Cite

show abstract

“…By nature, it is more about a way to improve the performance of existing networks from inside rather than an independent method for specific tasks. For example, it has been employed for improving the classification performance of RNNs [21] and CNNs [22,23]. It has also been applied to the time series prediction [24,25], to overcome the catastrophic interference in multi-task learning [26], and to improve the performance of cache-based communications [27].…”

Section: B Conceptor Learningmentioning

confidence: 99%

Conceptor Learning for Class Activation Mapping

Qian¹,

Yang²,

Zhang³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Class Activation Mapping (CAM) has been widely adopted to generate saliency maps which provides visual explanations for deep neural networks (DNNs). The saliency maps are conventionally generated by fusing the channels of the target feature map using a weighted average scheme. It is a weak model for the inter-channel relation, in the sense that it only models the relation among channels in a contrastive way (i.e., channels that play key roles in the prediction are given higher weights for them to stand out in the fusion). The collaborative relation, which makes the channels work together to provide cross reference, has been ignored. Furthermore, the model has neglected the intra-channel relation thoroughly. In this paper, we address this problem by introducing Conceptor learning into CAM generation. Conceptor leaning has been originally proposed to model the patterns of state changes in recurrent neural networks (RNNs). By relaxing the dependency of Conceptor learning to RNNs, we make Conceptor-CAM not only generalizable to more DNN architectures but also able to learn both the inter-and intra-channel relations for better saliency map generation. Moreover, we have enabled the use of Boolean operations to combine the positive and pseudo-negative evidences, which has made the CAM inference more robust and comprehensive. The effectiveness of Conceptor-CAM has been validated with both formal verifications and experiments on the dataset of the largest scale in literature. The experimental results show that Conceptor-CAM is compatible with and can bring significant improvement to all well recognized CAM-based methods, and has outperformed the state-of-the-art methods by 43.14% ∼ 72.79% (88.39% ∼ 168.15%) on ILSVRC2012 in Average Increase (Drop), 15.42% ∼ 42.55% (47.09% ∼ 372.09%) on VOC, and 17.43% ∼ 31.32% (47.54% ∼ 206.45%) on COCO, respectively.

show abstract

“…Conceptor [7] is a lately proposed tool for recurrent neural networks and can be understood as filters which characterise temporal neural activation patterns. Recently, its applications are extended to DNN and to deal with non-temporal data [8,9]. Usually conceptor takes the form of a projection matrix and achieves a direction-selective damping of high-dimensional network signals.…”

mentioning

confidence: 99%

End‐to‐end training algorithm for conceptor‐based neural networks

Qian

Zhang

2018

Electron. lett.

Self Cite

View full text Add to dashboard Cite

Since a conceptor, achieving direction-selective damping of high-dimensional network signals, usually takes the form of a projection matrix and is deduced analytically, a conceptor-based neural network is thought to be untrainable with backpropagation and gradient-descent algorithms from end to end. It limits the application of conceptors. To address this issue, an algorithm is proposed to train conceptor-based neural networks from end to end with gradient-descent algorithms. To the best of the authors' knowledge, it is the first work of such an end-to-end training algorithm. To develop this algorithm, a softmax-like loss function involved with conceptors is constructed empirically. Based on this loss function, corresponding gradients are deduced by using backpropagation method so that it is possible to train conceptor neural networks from end to end with a gradient-descent algorithm. Several experiments are conducted to show the feasibility and effectiveness of the proposed training algorithm.

show abstract

Fast Conceptor Classifier in Pre-trained Neural Networks for Visual Recognition

Cited by 4 publications

References 15 publications

Deep Learning Application to Detect Glaucoma with a Mixed Training Approach: Public Database and Expert-Labeled Glaucoma Population

Deep Learning Application to Detect Glaucoma with a Mixed Training Approach: Public Database and Expert-Labeled Glaucoma Population

Conceptor Learning for Class Activation Mapping

End‐to‐end training algorithm for conceptor‐based neural networks

Contact Info

Product

Resources

About