Multiobjective Evolutionary Optimization for Prototype-Based Fuzzy Classifiers

Abstract

Evolving intelligent systems (EISs), particularly, the zero-order ones have demonstrated strong performance on many real-world problems concerning data stream classification, while offering high model transparency and interpretability thanks to their prototype-based nature. Zero-order EISs typically learn prototypes by clustering streaming data online in a "one pass" manner for greater computation efficiency. However, such identified prototypes often lack optimality, resulting in less precise classification boundaries, thereby hindering the potential classification performance of the systems. To address this issue, a commonly adopted strategy is to minimise the training error of the models on historical training data or alternatively, to iteratively minimise the intra-cluster variance of the clusters obtained via online data partitioning. This recognises the fact that the ultimate classification performance of zero-order EISs is driven by the positions of prototypes in the data space. Yet, simply minimising the training error may potentially lead to overfitting, whilst minimising the intra-cluster variance does not necessarily ensure the optimised prototype-based models to attain improved classification outcomes. To achieve better classification performance whilst avoiding overfitting for zero-order EISs, this paper presents a novel multi-objective optimisation approach, enabling EISs to obtain optimal prototypes via involving these two disparate but complementary strategies simultaneously. Five decision-making schemes are introduced for selecting a suitable solution to deploy from the final non-dominated set of the resulting optimised models. Systematic experimental studies are carried out to demonstrate the effectiveness of the proposed optimisation approach in improving the classification performance of zero-order EISs.

Multilayer Evolving Fuzzy Neural Networks

“…3) SOTA Methods for Comparison: In this study, the following 17 single-model SOTA algorithms are used for performance comparison: 1) SVM [67]; 2) KNN [68]; 3) sequential classifier (SEQ) [69]; 4) sequence-dictionary-based KNN classifier (SDKNN) [69]; 5) extreme learning machine (ELM) [70]; 6) MLP; 7) LSTM [71]; 8) probabilistic neural network (PNN) [72]; 9) eigenvalue classifier (EIG) [73]; 10) spherical approximation classifier (SPA) [74]; 11) self-adaptive fuzzy learning system (SALF) [26]; 12) multi-objective optimised self-organising fuzzy inference system (MOOSOFIS) [75]; 13) SEFIS [38]; 14) ESAFIS [31]; 15) PALM [44]; 16) eClass0 classifier [24], and; 17) eClass1 classifier [24].…”

“…In running the experiments, for OD, PR and IS datasets, their original training-testing splits are used. For the other seven datasets, 50% of data samples are randomly selected to construct the training sets and the remaining 50% are used as the validation sets [75]. The detailed classification results in terms of accuracy (Acc) and balanced accuracy (BAcc) are given in Supplementary Tables S9 and S10, respectively.…”

“…The results obtained by MEFNN (in terms of average classification accuracy and standard deviation) and a selected group of the SOTA approaches from the literature are given in Table II for comparison. Note that the results by comparative approaches are obtained from [75], [82] directly. It can be seen from Table II that MEFNN is able to achieve great classification performance on these visual classification problems, outperforming many DNN-based approaches.…”

Metric networks for enhanced perception of non-local semantic information

“…Comprising two identical neural networks with shared parameters, siamese networks generate a similarity score through a distance metric function. They have proven successful in tasks such as image matching and face recognition (Gu et al, 2022b ; Li M. et al, 2022 ). Distance metric learning is a key aspect of metric learning, aiming to learn a function that can measure the distance between samples.…”

“…Distance metric learning is a key aspect of metric learning, aiming to learn a function that can measure the distance between samples. Various methods exist for distance metric learning, including prototype-based methods (Gu et al, 2022b ), metric matrix-based methods (Price et al, 2022 ), and maximum margin-based methods (Li X. et al, 2022 ). Among these, Max-Margin Metric Learning (MMML) has emerged as a classic technique maximizing the distances between different classes while minimizing the distances within the same class.…”