On Effects of Compression with Hyperdimensional Computing in Distributed Randomized Neural Networks

Abstract

A change of the prevalent supervised learning techniques is foreseeable in the near future: from the complex, computational expensive algorithms to more flexible and elementary training ones. The strong revitalization of randomized algorithms can be framed in this prospect steering. We recently proposed a model for distributed classification based on randomized neural networks and hyperdimensional computing, which takes into account cost of information exchange between agents using compression. The use of compression is important as it addresses the issues related to the communication bottleneck, however, the original approach is rigid in the way the compression is used. Therefore, in this work, we propose a more flexible approach to compression and compare it to conventional compression algorithms, dimensionality reduction, and quantization techniques.

A Survey on Hyperdimensional Computing aka Vector Symbolic Architectures, Part II: Applications, Cognitive Models, and Challenges

“…There are also efforts to apply HDC/VSA within machine learning outside of classification. Examples of such efforts are using data transformed into HVs for clustering [7,137], unsupervised learning [279,306], multi-task learning [32][33][34], distributed learning [149,361], model compression [32,143,361,362], and ensemble learning [25,409].…”