Visualization of Hidden Node Activity in Neural Networks: II. Application to RBF Networks

Duch, Włodzisław

doi:10.1007/978-3-540-24844-6_6

Cited by 7 publications

(3 citation statements)

References 5 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…An alternative to the rule-based understanding of the SVM function may be based on visualization techniques, as it has been done for MLP [10] and RBF neural networks [11,9].…”

Section: Discussionmentioning

confidence: 99%

Prototype Rules from SVM

Blachnik

Duch

2008

Rule Extraction From Support Vector Machines

Self Cite

View full text Add to dashboard Cite

“…An alternative to the rule-based understanding of the SVM function may be based on visualization techniques, as it has been done for MLP [10] and RBF neural networks [11,9].…”

Section: Discussionmentioning

confidence: 99%

Prototype Rules from SVM

Blachnik

Duch

2008

Rule Extraction From Support Vector Machines

Self Cite

View full text Add to dashboard Cite

“…The reason for this failure is rather simple: neural and other classifiers try to achieve linear separability, and non-linear separable data may require a non-trivial transformation that is very difficult to learn. Looking at the image of the training data in the space defined by the activity of the hidden layer neurons [8,9] one may notice that a perfect solution is frequently found in the hidden space -all data falls into separate clusters -but the clusters are non-separable, therefore the perceptron output layer is unable to provide useful results.…”

Section: Introductionmentioning

confidence: 99%

Separability is not the best goal for machine learning

Duch

2018

Preprint

View full text Add to dashboard Cite

Neural networks use their hidden layers to transform input data into linearly separable data clusters, with a linear or a perceptron type output layer making the final projection on the line perpendicular to the discriminating hyperplane. For complex data with multimodal distributions this transformation is difficult to learn. Projection on k ≥ 2 line segments is the simplest extension of linear separability, defining much easier goal for the learning process. Simple problems are 2-separable, but problems with inherent complex logic may be solved in a simple way by k-separable projections. The difficulty of learning nonlinear data distributions is shifted to separation of line intervals, simplifying the transformation of data by hidden network layers. For classification of difficult Boolean problems, such as the parity problem, linear projection combined with k-separability is sufficient and provides a powerful new target for learning. More complex targets may also be defined, changing the goal of learning from linear discrimination to creation of data distributions that can easily be handled by specialized models selected to analyze output distributions. This approach can replace many layers of transformation required by deep learning models.

show abstract

“…In our work, we not only visualize the weights along with the selected data, but convert the weight information and the statistics of the selected data into color and size representations for the input nodes. More recently, Duch [8,9] introduced a new projection on a lattice of hypercube nodes to visualize the hidden and output node activities in a high dimensional space. This method can be applied to any type of neural networks.…”

Section: Related Workmentioning

confidence: 99%

Opening the Black Box - Data Driven Visualization of Neural Networks

Tzeng¹,

Ma²

VIS 05. IEEE Visualization, 2005.

View full text Add to dashboard Cite

Artificial neural networks are computer software or hardware models inspired by the structure and behavior of neurons in the human nervous system. As a powerful learning tool, increasingly neural networks have been adopted by many large-scale information processing applications but there is no a set of well defined criteria for choosing a neural network. The user mostly treats a neural network as a black box and cannot explain how learning from input data was done nor how performance can be consistently ensured. We have experimented with several information visualization designs aiming to open the black box to possibly uncover underlying dependencies between the input data and the output data of a neural network. In this paper, we present our designs and show that the visualizations not only help us design more efficient neural networks, but also assist us in the process of using neural networks for problem solving such as performing a classification task. error bound, learning rate, training algorithm, hidden layer size, and the data vector used, are often chosen in a trial-and-error process.We believe visualization, which proves to help illustrate and understand the behaviors of complex systems, can also help us understand ANNs and design better ANNs. Previous attempts in using visualization to gain understanding into ANNs, as discussed in Section 3, mainly studied the weights and connections of a neural network and analyzed neural networks in isolation; the data used by the neural network were mostly not looked at.We therefore take a data-driven approach to the problem of visualizing ANN since gaining insights into a neural network requires the study of not only the network but also how it responds to the input data that it was designed to process. The methods we present enable the interactive exploration of both the input data and the neural network so as to gain more complete picture of how the neural network performs its task. The visualizations can also assist in the selection of network structure and other parameters for an assigned task, with the objectives to achieve better results and minimize the cost in terms of both space and time. Equally important is that the user can apply our visualization methods to study how neural networks use data, and gain further understanding into the potentially complex data relationships. In our work, we have applied visualization techniques to feed-forward neural networks trained with the back-propagation training algorithm [23], which is one of the most popular neural networks used for classification. Our designs and findings have helped us develop better intelligent visualization systems, and should also help others gain both understanding and confidence in using ANNs.

show abstract

Visualization of Hidden Node Activity in Neural Networks: II. Application to RBF Networks

Cited by 7 publications

References 5 publications

Prototype Rules from SVM

Prototype Rules from SVM

Separability is not the best goal for machine learning

Opening the Black Box - Data Driven Visualization of Neural Networks

Contact Info

Product

Resources

About