From Complex System Analysis to Pattern Recognition: Experimental Assessment of an Unsupervised Feature Extraction Method Based on the Relevance Index Metrics

Abstract

The so-called Relevance Index (RI) metrics are a set of recently-introduced indicators based on information theory principles that can be used to analyze complex systems by detecting the main interacting structures within them. Such structures can be described as subsets of the variables which describe the system status that are strongly statistically correlated with one another and mostly independent of the rest of the system. The goal of the work described in this paper is to apply the same principles to pattern recognition and check whether the RI metrics can also identify, in a high-dimensional feature space, attribute subsets from which it is possible to build new features which can be effectively used for classification. Preliminary results indicating that this is possible have been obtained using the RI metrics in a supervised way, i.e., by separately applying such metrics to homogeneous datasets comprising data instances which all belong to the same class, and iterating the procedure over all possible classes taken into consideration. In this work, we checked whether this would also be possible in a totally unsupervised way, i.e., by considering all data available at the same time, independently of the class to which they belong, under the hypothesis that the peculiarities of the variable sets that the RI metrics can identify correspond to the peculiarities by which data belonging to a certain class are distinguishable from data belonging to different classes. The results we obtained in experiments made with some publicly available real-world datasets show that, especially when coupled to tree-based classifiers, the performance of an RI metrics-based unsupervised feature extraction method can be comparable to or better than other classical supervised or unsupervised feature selection or extraction methods.

Exploring the Dynamic Organization of Random and Evolved Boolean Networks

“…In particular, we utilize the index called zI (see below), which makes it possible to identify, as components of a system, sets of variables that show a high degree of internal coordination. The RI methodology has been applied with interesting results to several systems: some of them had been artificially designed in order to test the effectiveness of the technique, while others referred to interesting physical, chemical, biological, or socio-economic systems [13][14][15][16].…”

“…Different methods to explore the dynamical organization of complex systems can be proposed. In this paper, we make use of one such method, which was introduced by us some years ago [10,11] and which has later been improved [12] and applied to different classes of systems [13][14][15][16]. The method identifies integrated groups of variables, by means of measures based upon information theory (the Relevance Index methodology-RI in the following), which is briefly recalled in Section 2.1 (referring the interested reader to the literature [12] for further details).…”