Deep Neural Fuzzy System Oriented toward High-Dimensional Data and Interpretable Artificial Intelligence

Abstract

Fuzzy systems (FSs) are popular and interpretable machine learning methods, represented by the adaptive neuro-fuzzy inference system (ANFIS). However, they have difficulty dealing with high-dimensional data due to the curse of dimensionality. To effectively handle high-dimensional data and ensure optimal performance, this paper presents a deep neural fuzzy system (DNFS) based on the subtractive clustering-based ANFIS (SC-ANFIS). Inspired by deep learning, the SC-ANFIS is proposed and adopted as a submodule to construct the DNFS in a bottom-up way. Through the ensemble learning and hierarchical learning of submodules, DNFS can not only achieve faster convergence, but also complete the computation in a reasonable time with high accuracy and interpretability. By adjusting the deep structure and the parameters of the DNFS, the performance can be improved further. This paper also performed a profound study of the structure and the combination of the submodule inputs for the DNFS. Experimental results on five regression datasets with various dimensionality demonstrated that the proposed DNFS can not only solve the curse of dimensionality, but also achieve higher accuracy, less complexity, and better interpretability than previous FSs. The superiority of the DNFS is also validated over other recent algorithms especially when the dimensionality of the data is higher. Furthermore, the DNFS built with five inputs for each submodule and two inputs shared between adjacent submodules had the best performance. The performance of the DNFS can be improved by distributing the features with high correlation with the output to each submodule. Given the results of the current study, it is expected that the DNFS will be used to solve general high-dimensional regression problems efficiently with high accuracy and better interpretability.

Fuzzy C-Means Clustering Based Deep Patch Learning With Improved Interpretability for Classification Problems

“…Therefore, this combination was chosen to be our parameter sharing strategy in the following experiments. Compared HI-FCMDPL-CP1 with HI-FCMDPL-CP2, we can conclude that PS improved the classification performance [36], [37] regardless of whether MIC and RI were used or not.…”

“…Though cascaded fuzzy system (CFS) and cascaded centralized TSK fuzzy system (CCTSKFS) has obvious superiority over the conventional FSs in approximation accuracy, robustness and interpretability [34], there are still problems such as rule explosion, poor interpretability and complex structure [35]. Our recently research in [36], [37] showed that parameter sharing (PS) and random input (RI) outperformed ANFIS in training Takagi-Sugeuo-Kang (TSK) fuzzy regression models. So, PS and RI are still used in our proposed approach.…”

Deep patch learning algorithms with high interpretability for regression problems

“…Our previous research in Chen et al 50 shows that the ways of feature input have a significant impact on algorithm performance. The SOTA methods (i.e., DCFS, 48 iDCFS 51 ) utilized the whole feature space of the data to construct their models, leading to lengthy rules for high-dimensional data and degradation of interpretability.…”

“…However, they are all generated based on the initial global model (and hence in series with the global model), and then, the model to be used is identified. Compared with existing approaches, 50,54 the new approach involves (1) an improved PL (iPL) is developed by fuzzy c-means (FCM) clustering 55 to significantly decrease the fuzzy rules and computational load; (2) two hierarchical (deeper-learned) iPL FSs are constructed; (3) The iPL is enhanced in DPLFSs by replacing the shared membership functions (MFs) with independent ones in adaptive network-based fuzzy inference system (ANFIS), and hence to decouple the number of rules from the number of MFs in each input domain, and (4) PL only considered low dimensional in 1D/2D/3D regression problems, PL is extended to high dimensional space in this paper; (5) PCC, MIC and principal components analysis (PCA) are adopted as an automated feature extraction algorithm to optimize deep DPLFSs.…”

Design and Optimization of a Neuro-Fuzzy System for the Control of an Electromechanical Plant

“…It is possible to use training algorithms with neural networks when representing the fuzzy system as a neural network that grants higher accuracy in the fuzzy system performance. When representing a fuzzy system as a neural network, a neuro-fuzzy system is obtained [4,5].…”