Visualization of maximizing images with deconvolutional optimization method for neurons in deep neural networks

Nekhaev, Dmitry; Демин, В. А.

doi:10.1016/j.procs.2017.11.174

Cited by 6 publications

(8 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It is worth noting that the simple multiplication of feed-forward weight matrices gave quite recognizable visualizations of digits learned by the output neurons (the 2nd row in Figure 5 ). Slightly better and less noisy results were obtained by the method described in Nekhaev and Demin ( 2017 ) (the 1st row in Figure 5 ). It was also shown that the forward and reciprocal weights between the hidden and output layers were highly correlated.…”

Section: Resultsmentioning

confidence: 86%

See 1 more Smart Citation

Recurrent Spiking Neural Network Learning Based on a Competitive Maximization of Neuronal Activity

Демин

Nekhaev

2018

Front. Neuroinform.

Self Cite

View full text Add to dashboard Cite

Spiking neural networks (SNNs) are believed to be highly computationally and energy efficient for specific neurochip hardware real-time solutions. However, there is a lack of learning algorithms for complex SNNs with recurrent connections, comparable in efficiency with back-propagation techniques and capable of unsupervised training. Here we suppose that each neuron in a biological neural network tends to maximize its activity in competition with other neurons, and put this principle at the basis of a new SNN learning algorithm. In such a way, a spiking network with the learned feed-forward, reciprocal and intralayer inhibitory connections, is introduced to the MNIST database digit recognition. It has been demonstrated that this SNN can be trained without a teacher, after a short supervised initialization of weights by the same algorithm. Also, it has been shown that neurons are grouped into families of hierarchical structures, corresponding to different digit classes and their associations. This property is expected to be useful to reduce the number of layers in deep neural networks and modeling the formation of various functional structures in a biological nervous system. Comparison of the learning properties of the suggested algorithm, with those of the Sparse Distributed Representation approach shows similarity in coding but also some advantages of the former. The basic principle of the proposed algorithm is believed to be practically applicable to the construction of much more complicated and diverse task solving SNNs. We refer to this new approach as “Family-Engaged Execution and Learning of Induced Neuron Groups,” or FEELING.

show abstract

Section: Resultsmentioning

confidence: 86%

“… Last layer weights visualization. The first row contains reconstructed maximizing images for the output neurons (Nekhaev and Demin, 2017 ). The second row is a simple product of two forward weight matrices: one of the size 784 × 100 and the other of size 100 × 10.…”

Section: Resultsmentioning

confidence: 99%

Recurrent Spiking Neural Network Learning Based on a Competitive Maximization of Neuronal Activity

Демин

Nekhaev

2018

Front. Neuroinform.

Self Cite

View full text Add to dashboard Cite

show abstract

“…Various studies have been conducted in the literature that deals with the subject of artificial neural networks and graph. Nekhaev & Demin (2017) visualized the neural network and its hidden layers to solve the problem of the incomprehensibility of the operation of deep artificial neural networks. Liu (2017) significantly reduced the search time by performing the searches on the neural network they dealt with in the article they published on the graph structure.…”

Section: Methodsmentioning

confidence: 99%

The Use of Graph Databases for Artificial Neural Networks

Özdemir

Kinaci

2021

Journal of Advanced Research in Natural and Applied Sciences

View full text Add to dashboard Cite

Storing and using trained artificial neural network (ANN) models face technical difficulties. These models are usually stored as files and cannot be run directly. An artificial neural network can be structurally expressed as a graph. Therefore, it would be much more useful to store ANN models in a database and use the graph database as this database system. In this study, training and testing stages of ANN models are provided with software that will allow multiple researchers to conduct joint research on ANN models. The developed software platform is aimed to increase the representation power of the currently used methods by transferring the models developed in the popular ANN frameworks used today. With the study conducted, even someone who has started learning artificial neural network models from scratch will see the process and can visually develop their own model. When models are stored in the graph database, it will be easier to making versions and observing how the model grows. In addition, data to be input and output to the model can be stored in this database, also. In order to feed ANN models with input data and produce outputs, the graph database's own query language was used. This eliminates the dependency on another software library.

show abstract

“…In addition, many saliency methods are criticized recently for giving misleading visualization interpretations and researchers are advised to use them with caution [34]. To unveil the CNN models further, the direct deconvolution and indirect optimization are the two major approaches [35]. Deconvolution starts with finding an image from the dataset that triggers high activity to the neuron of interest and the gradient of neuron activity is calculated [35].…”

Section: Visualization Of Cnn Modelsmentioning

confidence: 99%

“…To unveil the CNN models further, the direct deconvolution and indirect optimization are the two major approaches [35]. Deconvolution starts with finding an image from the dataset that triggers high activity to the neuron of interest and the gradient of neuron activity is calculated [35]. In general, a deconvolutional network is a reversed convolutional network, which maps features back to pixels [36].…”

Section: Visualization Of Cnn Modelsmentioning

confidence: 99%

Deep Learning and Its Applications in Computational Pathology

Hong¹,

Fenyö²

2022

Preprint

View full text Add to dashboard Cite

Deep learning techniques, such as convolutional neural networks (CNN), generative adversarial networks (GAN), and graph neural networks (GNN), have over the past decade changed the ac-curacy of prediction in many diverse fields. In recent years, the application of deep learning tech-niques in computer vision tasks in pathology demonstrated extraordinary potential in assisting clinicians, automating diagnosis, and reducing costs for patients. Formerly unknown pathologi-cal evidence, such as morphological features related to specific biomarkers, copy number varia-tions, and other molecular features, were also able to be captured by deep learning models. In this paper, we review popular deep learning methods and some recent publications about their appli-cations in pathology.

show abstract

Visualization of maximizing images with deconvolutional optimization method for neurons in deep neural networks

Cited by 6 publications

References 6 publications

Recurrent Spiking Neural Network Learning Based on a Competitive Maximization of Neuronal Activity

Recurrent Spiking Neural Network Learning Based on a Competitive Maximization of Neuronal Activity

The Use of Graph Databases for Artificial Neural Networks

Deep Learning and Its Applications in Computational Pathology

Contact Info

Product

Resources

About