A new approach for data clustering and visualization using self-organizing maps

2016

ijarai

Abstract-In this paper, we introduce a new type of information-theoretic method called "information-theoretic active SOM", based on the self-organizing maps (SOM) for training multi-layered neural networks. The SOM is one of the most important techniques in unsupervised learning. However, SOM knowledge is sometimes ambiguous and cannot be easily interpreted. Thus, we introduce the information-theoretic method to produce clearer and interpretable representations. The present method extends this information-theoretic approach into supervised learning. The main contribution can be summarized by three points. First, it is shown that clear representations by the information-theoretic method can be effective in training supervised learning. Second, the method is sufficiently simple where there are two separated components, namely, information maximization and error minimization component. Usually, two components are mixed in one framework, and it is difficult to compromise between them. In addition, the knowledge obtained by this information-theoretic SOM can be used to solve the shortage of unlabeled data, because the information maximization component is unsupervised and can process all input data with and without labels. The method was applied to the well-known image segmentation datasets. Experimental results showed that clear weights were produced and generalization performance was improved by using the information-theoretic SOM. In addition, the final results were stable, almost independent of the parameter values.

Section: A Explicit Knowledge For Supervised Learningmentioning

confidence: 99%

Information-Theoretic Active SOM for Improving Generalization Performance

2016

ijarai

“…In SOM, much attention has been paid in particular to topological preservation, and many methods to measure topological consistency have been proposed [12], [13], [14], [15], [16], [17], [18]. In addition, many visualization methods have also been developed to interpret the SOM knowledge obtained by learning [19], [20], [21], [22], [23], [24], [25], [26], [27], [28], [29]. However, in spite of having a good reputation for visualization, SOM has faced difficulty in visualizing results obtained by learning.…”

Section: B Information-theoretic Sommentioning

confidence: 99%

“…For visual evaluation, we used the principal component analysis (PCA) to summarize connection weights. As mentioned in the introduction section, there is difficulty in interpreting SOM knowledge, a number of methods have been developed to clarify the knowledge [20], [21], [22], [23], [24], [25], [26], [27], [28], [29]. In this study, we used the PCA for clarification, in particular for simplifying the knowledge.…”

Section: A Experiments Outlinementioning

confidence: 99%

Double Competition for Information-Theoretic SOM

2014

ijarai

Abstract-In this paper, we propose a new type of informationtheoretic method for the self-organizing maps (SOM), taking into account competition between competitive (output) neurons as well as input neurons. The method is called "double competition", as it considers competition between outputs as well as input neurons. By increasing information in input neurons, we expect to obtain more detailed information on input patterns through the information-theoretic method. We applied the informationtheoretic methods to two well-known data sets from the machine learning database, namely, the glass and dermatology data sets. We found that the information-theoretic method with double competition explicitly separated the different classes. On the other hand, without considering input neurons, class boundaries could not be explicitly identified. In addition, without considering input neurons, quantization and topographic errors were inversely related. This means that when the quantization errors decreased, topographic errors inversely increased. However, with double competition, this inverse relation between quantization and topographic errors was neutralized. Experimental results show that by incorporating information in input neurons, class structure could be clearly identified without degrading the map quality to severely.

“…Though the SOM has a good reputation in visualization and interpretation, we have had serious difficulty in extracting useful information from the knowledge obtained. Thus, a number of different types of methods have been developed to clarify SOM knowledge, [6], [7], [8], [9], [10], [11], [12]. In addition, there have been many other methods to create more interpretable connection weights by changing the learning procedures [13], [14], [15], [16].…”

Section: A Contradiction Resolutionmentioning

confidence: 99%

Contradiction Resolution of Competitive and Input Neurons to Improve Prediction and Visualization Performance

2013

IJARAI

Abstract-In this paper, we propose a new type of informationtheoretic method to resolve the contradiction observed in competitive and input neurons. For competitive neurons, contradiction between self-evaluation (individuality) and outer-evaluation (collectivity) exists, which is reduced to realize the self-organizing maps. For input neurons, there exists contradiction between the use of many and few input neurons. We try to realize a situation where as many input neurons as possible are used, and at the same time, another where only a few input neurons are used. This contradictory situation can be resolved by viewing input neurons on different levels, namely, the individual and average level. We applied contradiction resolution to two data sets, namely, the Japanese short term economy survey (Tankan) and Dollar-Yen exchange rates. In both data sets, we succeeded in improving the prediction performance. Many input neurons were used on average, but a few input neurons were only taken for each input pattern. In addition, connection weights were condensed into a small number of distinct groups for better prediction and interpretation performance.