Using the correlation criterion to position and shape RBF units for incremental modelling

Abstract

A novel technique is proposed for the incremental construction of sparse radial basis function (RBF) networks. The correlation between a RBF regressor and the training data is used as the criterion to position and shape the RBF node, and it is shown that this is equivalent to incrementally minimise the modelling mean square error. A guided random search optimisation method, called the repeated weighted boosting search, is adopted to append RBF nodes one by one in an incremental regression modelling procedure. The experimental results obtained using the proposed method demonstrate that it provides a viable alternative to the existing state-of-the-art modelling techniques for constructing parsimonious RBF models that generalise well.

Self-supervised Knowledge Distillation Using Singular Value Decomposition

“…RBF is a function that re-maps each feature in a viewpoint of distance from the center so that the feature has a high dimension. RBF can be used for various kernelized learning or RBF network (RBFN) [14,15]. In particular, analyzing features with RBF such as Gaussian function makes it possible to analyze noisy data more robustly.…”

“…Since the derived feature information from a singular vector is generally noisy, noise-robust methods are required. Therefore, we employ Gaussian RBF, which is a frequently used kernel function for analyzing noisy data [14,15], as a way to obtain the correlation.…”

“…So we gracefully compress the knowledge data by utilizing singular value decomposition (SVD), which is mainly applied to dimension reduction of features [11,12,13] in signal processing domain. We also analyze the correlation between compressed feature maps through a radial basis function (RBF) [14,15], which is often used for kernelized learning. As a result, knowledge distillation using SVD and RBF can distill the information of T-DNN more efficiently than conventional techniques, and can transfer regardless of the spatial resolution of feature maps.…”