An Efficient Algorithm for the Incremental Broad Learning System by Inverse Cholesky Factorization of a Partitioned Matrix

Zhu, Hufei; Liu, Zhulin; Chen, C. L. Philip; Liang, Yanyang

doi:10.1109/access.2021.3052102

Cited by 4 publications

(3 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Moreover, a foreseeable challenge in integrating multifeatures into Word-unit BLS is the substantial increase in training time. One potential solution to this challenge is the Zhu method [46], which reduces computational complexity by utilizing the inverse Cholesky factor of the Hermitian matrix during pseudoinverse computation and factor updates. Despite the limitations of the study, this paper introduced a novel framework that integrates BLS with linguistic theory.…”

Section: B Limitationsmentioning

confidence: 99%

Enhanced Word-Unit Broad Learning System With Sememes

Jiang,

Lu,

et al. 2023

IEEE Access

View full text Add to dashboard Cite

High accuracy in text classification can be achieved by simultaneously learning multiple sources of information, such as sequence and word. In this study, we propose a novel learning framework for text classification, called Word-unit Broad Learning System (BLS). The Word-unit BLS utilises a flat neural network known as BLS and offers three key advantages. First, it provides higher accuracy and shorter training time compared to popular machine learning methods, allowing for the simultaneous learning of sequence information and word importance. Second, we incorporate a multi-layer perceptron with attention aggregation in the feature-mapped layer, along with position encoding, to capture the latent relationship between each word and the contextual information in a global context. Lastly, we introduce a novel approach to enhance word representation by employing sememes in the enhancement node layer, thereby improving the feature distribution of each word in the vector space. The effectiveness of the proposed framework was evaluated by conducting experiments on four datasets covering various types of text classifications. The results demonstrate that Word-unit BLS achieves 8.26% higher accuracy than Naive Bayes while requiring 1 33 ⁄ of the training time. Furthermore, when compared with traditional BLS models, Word-unit BLS outperforms in learning the sequence information. The effectiveness of sememe enhancement in word representation is also demonstrated, particularly in the case of large-scale datasets.

show abstract

Section: B Limitationsmentioning

confidence: 99%

Enhanced Word-Unit Broad Learning System With Sememes

Jiang,

Lu,

et al. 2023

IEEE Access

View full text Add to dashboard Cite

show abstract

“…To reduce the computational complexity, the recursive BLS algorithm and the square-root BLS algorithm proposed in [12] compute the output weights from the inverse and the inverse Cholesky factor of the Hermitian matrix in the ridge inverse, respectively, which are usually smaller than the ridge inverse. On the other hand, the square-root BLS algorithms on added nodes have been proposed in [13], [14] to improve the original BLS on added nodes in [7]. The square-root BLS algorithm proposed in [14] still computes the generalized inverse solution for the output weights, while in [13], the square-root BLS algorithm has been proposed to compute the ridge solution for the output weights from the inverse Cholesky factor of the Hermitian matrix in the ridge inverse.…”

Section: Introductionmentioning

confidence: 99%

“…On the other hand, the square-root BLS algorithms on added nodes have been proposed in [13], [14] to improve the original BLS on added nodes in [7]. The square-root BLS algorithm proposed in [14] still computes the generalized inverse solution for the output weights, while in [13], the square-root BLS algorithm has been proposed to compute the ridge solution for the output weights from the inverse Cholesky factor of the Hermitian matrix in the ridge inverse.…”

Section: Introductionmentioning

confidence: 99%

Low-Memory Implementations of Ridge Solutions for Broad Learning System with Incremental Learning

Zhu

2021

Preprint

Self Cite

View full text Add to dashboard Cite

The existing low-memory BLS implementation proposed recently avoids the need for storing and inverting large matrices, to achieve efficient usage of memories. However, the existing low-memory BLS implementation sacrifices the testing accuracy as a price for efficient usage of memories, since it can no longer obtain the generalized inverse or ridge solution for the output weights during incremental learning, and it cannot work under the very small ridge parameter (i.e., λ = 10 −8 ) that is utilized in the original BLS. Accordingly, it is required to develop the low-memory BLS implementations, which can work under very small ridge parameters and compute the generalized inverse or ridge solution for the output weights in the process of incremental learning.In this paper, firstly we propose the low-memory implementations for the recently proposed recursive and square-root BLS algorithms on added inputs and the recently proposed squareroot BLS algorithm on added nodes, by simply processing a batch of inputs or nodes in each recursion. Since the recursive BLS implementation includes the recursive updates of the inverse matrix that may introduce numerical instabilities after a large number of iterations, and needs the extra computational load to decompose the inverse matrix into the Cholesky factor when cooperating with the proposed low-memory implementation of the square-root BLS algorithm on added nodes, we only improve the low-memory implementations of the square-root BLS algorithms on added inputs and nodes, to propose the full lowmemory implementation of the square-root BLS algorithm.All the proposed low-memory BLS implementations compute the ridge solution for the output weights in the process of incremental learning, and most of them can work under very small ridge parameters. When the ridge parameter is not too small, the proposed low-memory implementations for the recursive and square-root BLS algorithms on added inputs and the part for added inputs of the proposed full low-memory BLS implementation usually achieve better testing accuracies than the existing low-memory BLS implementation on added inputs. More importantly, when the ridge parameter is very small (i.e., λ = 10 −8 ) as in the original BLS, the existing low-memory BLS implementation on added inputs cannot work in any update (of the incremental learning), the proposed low-memory implementation for the recursive BLS algorithm on added inputs cannot work in the last update, while the proposed low-memory implementation for the square-root BLS algorithm on added inputs and the proposed full low-memory implementation of the square-root BLS algorithm (on added inputs and nodes) can work in all updates.With respect to the existing low-memory BLS implementation on added inputs, the proposed low-memory implementation for the recursive BLS algorithm on added inputs and the part for added inputs of the proposed full low-memory implementation of the square-root BLS algorithm require nearly the same training time, while the proposed low-memory implementation for ...

show abstract

Telescopic broad Bayesian learning for big data stream

Yuen,

Kuok

2024

Computer aided Civil Eng

View full text Add to dashboard Cite

In this paper, a novel telescopic broad Bayesian learning (TBBL) is proposed for sequential learning. Conventional broad learning suffers from the singularity problem induced by the complexity explosion as data are accumulated. The proposed TBBL successfully overcomes the challenging issue and is feasible for sequential learning with big data streams. The learning network of TBBL is reconfigurable to adopt network augmentation and condensation. As time evolves, the learning network is augmented to incorporate the newly available data and additional network components. Meanwhile, the learning network is condensed to eliminate the network connections and components with insignificant contributions. Moreover, as a benefit of Bayesian inference, the uncertainty of the estimates can be quantified. To demonstrate the efficacy of the proposed TBBL, the performance on highly nonstationary piecewise time series and complex multivariate time series with 100 million data points are presented. Furthermore, an application for long‐term structural health monitoring is presented.

show abstract

An Efficient Algorithm for the Incremental Broad Learning System by Inverse Cholesky Factorization of a Partitioned Matrix

Cited by 4 publications

References 32 publications

Enhanced Word-Unit Broad Learning System With Sememes

Enhanced Word-Unit Broad Learning System With Sememes

Low-Memory Implementations of Ridge Solutions for Broad Learning System with Incremental Learning

Telescopic broad Bayesian learning for big data stream

Contact Info

Product

Resources

About