RCBLS: An Outlier-Robust Broad Learning Framework with Compact Structure

Abstract

Recently, the broad learning system (BLS) has been widely developed in virtue of its excellent performance and high-computational efficiency. However, two deficiencies still exist in BLS and preclude its deployment in real applications. First, the standard BLS performs poorly in outlier environments because the least squares loss function it uses to train the network is sensitive to outliers. Second, the model structure of BLS is likely to be redundant since the hidden nodes in it are randomly generated. To address the above two issues, a new robust and compact BLS (RCBLS), based on M-estimator and sparsity regularization, is proposed in this paper. The RCBLS develops from the BLS model and maintains its excellent characteristics, but replaces the conventional least squares learning criterion with an M-estimator-based loss function that is less sensitive to outliers, in order to suppress the incorrect feedback of the model to outlier samples, and hence enhance its robustness in the presence of outliers. Meanwhile, the RCBLS imposes the sparsity-promoting l2,1 -norm regularization instead of the common l2-norm regularization for model reduction. With the help of the row sparsity of l2,1-norm regularization, the unnecessary hidden nodes in RCBLS can be effectively picked out and removed from the network, thereby resulting in a more compact network. The theoretical analyses on outlier robustness, structural compactness and computational complexity of the proposed RCBLS model are provided. Finally, the validity of the RCBLS is verified by regression, time series prediction and image classification tasks. The experimental results demonstrate that the proposed RCBLS has stronger anti-outlier ability and more compact network structure than BLS and other representative algorithms.