End to End Binarized Neural Networks for Text Classification

Shridhar, Kumar; Jain, Harshil; Agarwal, Akshat; Kleyko, Denis

doi:10.18653/v1/2020.sustainlp-1.4

Cited by 15 publications

(12 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The works in this domain dealt with different tasks such as text categorization, news identification, and intent classification. Most of the works [Rachkovskij, 2007], [Shridhar et al, 2020], [Alonso et al, 2021] used HVs as a way to represent data for some conventional machine learning classification algorithms. 1.…”

Section: Classification Based On Feature Vectorsmentioning

confidence: 99%

A Survey on Hyperdimensional Computing aka Vector Symbolic Architectures, Part II: Applications, Cognitive Models, and Challenges

Kleyko,

Rachkovskij,

Osipov

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

This is Part II of the two-part comprehensive survey devoted to a computing framework most commonly known under the names Hyperdimensional Computing and Vector Symbolic Architectures (HDC/VSA). Both names refer to a family of computational models that use high-dimensional distributed representations and rely on the algebraic properties of their key operations to incorporate the advantages of structured symbolic representations and vector distributed representations. Holographic Reduced Representations [Plate, 1995], [Plate, 2003] is an influential HDC/VSA model that is well-known in the machine learning domain and often used to refer to the whole family. However, for the sake of consistency, we use HDC/VSA to refer to the area.Part I of this survey [Kleyko et al., 2021c] covered foundational aspects of the area, such as historical context leading to the development of HDC/VSA, key elements of any HDC/VSA model, known HDC/VSA models, and transforming input data of various types into high-dimensional vectors suitable for HDC/VSA. This second part surveys existing applications, the role of HDC/VSA in cognitive computing and architectures, as well as directions for future work. Most of the applications lie within the machine learning/artificial intelligence domain, however we also cover other applications to provide a thorough picture. The survey is written to be useful for both newcomers and practitioners.

show abstract

Section: Classification Based On Feature Vectorsmentioning

confidence: 99%

A Survey on Hyperdimensional Computing aka Vector Symbolic Architectures, Part II: Applications, Cognitive Models, and Challenges

Kleyko,

Rachkovskij,

Osipov

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…The most illustrative use cases for encoding of input data into hypervectors and interfacing convectional machine learning algorithms are [34], [47]- [52]. For example, works [48], [50] proposed encoding n-gram statistics into hypervectors and subsequently solving typical natural language processing tasks with either supervised or unsupervised learning using standard artificial neural network architectures.…”

Section: Related Workmentioning

confidence: 99%

HyperSeed: Unsupervised Learning with Vector Symbolic Architectures

Osipov¹,

Kahawala²,

Haputhanthri³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

Motivated by recent innovations in biologicallyinspired neuromorphic hardware, this paper presents a novel unsupervised machine learning approach named Hyperseed that leverages Vector Symbolic Architectures (VSA) for fast learning a topology preserving feature map of unlabelled data. It relies on two major capabilities of VSAs: the binding operation and computing in superposition. In this paper, we introduce the algorithmic part of Hyperseed expressed within Fourier Holographic Reduced Representations VSA model, which is specifically suited for implementation on spiking neuromorphic hardware. The two distinctive novelties of the Hyperseed algorithm are: 1) Learning from only few input data samples and 2) A learning rule based on a single vector operation. These properties are demonstrated on synthetic datasets as well as on illustrative benchmark usecases, IRIS classification and a language identification task using n-gram statistics.

show abstract

“…The experiments reported in this paper were done with two approaches to forming network's classifier: regularized least squares (RLS) and centroids. Though, other alternatives are available [1,5,22,38]. The activations of the hidden layer of the network were formed according to a recently proposed version of RVFL network [25], which simplifies the conventional architecture [16] using some of the HDC/VSA principles [24,27].…”

Section: Introductionmentioning

confidence: 99%

On Effects of Compression with Hyperdimensional Computing in Distributed Randomized Neural Networks

Rosato

Panella

Osipov

et al. 2021

Advances in Computational Intelligence

Self Cite

View full text Add to dashboard Cite

A change of the prevalent supervised learning techniques is foreseeable in the near future: from the complex, computational expensive algorithms to more flexible and elementary training ones. The strong revitalization of randomized algorithms can be framed in this prospect steering. We recently proposed a model for distributed classification based on randomized neural networks and hyperdimensional computing, which takes into account cost of information exchange between agents using compression. The use of compression is important as it addresses the issues related to the communication bottleneck, however, the original approach is rigid in the way the compression is used. Therefore, in this work, we propose a more flexible approach to compression and compare it to conventional compression algorithms, dimensionality reduction, and quantization techniques.

show abstract

End to End Binarized Neural Networks for Text Classification

Cited by 15 publications

References 9 publications

A Survey on Hyperdimensional Computing aka Vector Symbolic Architectures, Part II: Applications, Cognitive Models, and Challenges

A Survey on Hyperdimensional Computing aka Vector Symbolic Architectures, Part II: Applications, Cognitive Models, and Challenges

HyperSeed: Unsupervised Learning with Vector Symbolic Architectures

On Effects of Compression with Hyperdimensional Computing in Distributed Randomized Neural Networks

Contact Info

Product

Resources

About