Finite precision implementation of random vector functional-link networks

Rosato, Antonello; Altilio, Rosa; Panella, Massimo

doi:10.1109/icdsp.2017.8096056

Cited by 8 publications

(9 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In the previous experiments, the weights in W out being compressed were realvalued since the proposed compression approach is real-valued as well. It is, however, known that neural networks can perform well even when the weights are limited to a few quantization levels [15,35]. Therefore, we did an experiment to demonstrate what to potentially expect for the quantization of W out .…”

Section: Quantization Of the Classifiermentioning

confidence: 99%

On Effects of Compression with Hyperdimensional Computing in Distributed Randomized Neural Networks

Rosato

Panella

Osipov

et al. 2021

Advances in Computational Intelligence

Self Cite

View full text Add to dashboard Cite

A change of the prevalent supervised learning techniques is foreseeable in the near future: from the complex, computational expensive algorithms to more flexible and elementary training ones. The strong revitalization of randomized algorithms can be framed in this prospect steering. We recently proposed a model for distributed classification based on randomized neural networks and hyperdimensional computing, which takes into account cost of information exchange between agents using compression. The use of compression is important as it addresses the issues related to the communication bottleneck, however, the original approach is rigid in the way the compression is used. Therefore, in this work, we propose a more flexible approach to compression and compare it to conventional compression algorithms, dimensionality reduction, and quantization techniques.

show abstract

Section: Quantization Of the Classifiermentioning

confidence: 99%

On Effects of Compression with Hyperdimensional Computing in Distributed Randomized Neural Networks

Rosato

Panella

Osipov

et al. 2021

Advances in Computational Intelligence

Self Cite

View full text Add to dashboard Cite

show abstract

“…In the previous experiments, the weights in W out being compressed were realvalued since the proposed compression approach is real-valued as well. It is, however, known that neural networks can perform well even when the weights are limited to a few quantization levels [42,43]. Therefore, we did an experiment to demonstrate what to potentially expect for the quantization of W out .…”

Section: Quantization Of the Classifiermentioning

confidence: 99%

On Effects of Compression with Hyperdimensional Computing in Distributed Randomized Neural Networks

Rosato¹,

Panella²,

Osipov³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Here, both W in and b are generated from a uniform distribution. Following [8], the range for W in is [−1, 1] while the range for b is [−0.1, 0.1]. Since W in and b are fixed the process of training RVFL is focused on learning the values of the readout matrix W out .…”

Section: A Random Vector Functional Linkmentioning

confidence: 99%

“…The second scenario compares the results for the real-valued readout matrix against the considered strategies of obtaining integer-valued readout matrix. The final scenario compares FPGA implementations of the proposed approach and the finite precision RVFL [8] in the case of a limited energy budget. All reported results are based on 121 real-world classification datasets obtained from the UCI Machine Learning Repository 7 .…”

Section: Performance Evaluationmentioning

confidence: 99%

“…Moreover, the average accuracy for the conventional RVFL is 0.76 while that for the proposed approach is 0.80. 8 It is not absolutely intuitive why the proposed approach demonstrates higher accuracy. Nevertheless, one hypothesis is that the quantization for the density-based encoding might provide extra regularization.…”

Section: A Comparison With the Conventional Rvflmentioning

confidence: 99%

See 1 more Smart Citation

Density Encoding Enables Resource-Efficient Randomly Connected Neural Networks

Kleyko,

Kheffache,

Frady

et al. 2019

Preprint

View full text Add to dashboard Cite

The deployment of machine learning algorithms on resource-constrained edge devices is an important challenge from both theoretical and applied points of view. In this article, we focus on resource-efficient randomly connected neural networks known as Random Vector Functional Link (RVFL) networks since their simple design and extremely fast training time make them very attractive for solving many applied classification tasks. We propose to represent input features via the densitybased encoding known in the area of stochastic computing and use the operations of binding and bundling from the area of hyperdimensional computing for obtaining the activations of the hidden neurons. Using a collection of 121 real-world datasets from the UCI Machine Learning Repository, we empirically show that the proposed approach demonstrates higher average accuracy than the conventional RVFL. We also demonstrate that it is possible to represent the readout matrix using only integers in a limited range with minimal loss in the accuracy. In this case, the proposed approach operates only on small n-bits integers, which results in a computationally efficient architecture. Finally, through hardware FPGA implementations, we show that such an approach consumes approximately eleven times less energy than that of the conventional RVFL.

show abstract

Finite precision implementation of random vector functional-link networks

Cited by 8 publications

References 27 publications

On Effects of Compression with Hyperdimensional Computing in Distributed Randomized Neural Networks

On Effects of Compression with Hyperdimensional Computing in Distributed Randomized Neural Networks

On Effects of Compression with Hyperdimensional Computing in Distributed Randomized Neural Networks

Density Encoding Enables Resource-Efficient Randomly Connected Neural Networks

Contact Info

Product

Resources

About