Estimation of Vectors Similarity by Their Randomized Binary Projections

Rachkovskij, Dmitri A.

doi:10.1007/s10559-015-9774-1

Cited by 11 publications

(10 citation statements)

References 45 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We have also been investigating the use of VSA vectors for both role and filler vectors that can be generated from a semantic vector space, for example Word2Vec, 28 via the method of randomised binary projection. 29 This enables semantically similar descriptions of radio's to be created allowing the discovery of similar services from different MDO partners.…”

Section: Representing Radio's As Vsa Vectorsmentioning

confidence: 99%

Energy efficient 'in memory' computing to enable decentralised service workflow composition in support of multi-domain operations

Bent

Simpkin

Taylor

et al. 2021

Artificial Intelligence and Machine Learning for Multi-Domain Operations Applications III

View full text Add to dashboard Cite

Future Multi-Domain Operations (MDO) will require the coordination of hundreds-even thousands-of devices and component services. This will demand the capability to rapidly discover the distributed devices/services and combine them into different workflow configurations, thereby creating the applications necessary to support changing mission needs. To meet these objectives, we envision a distributed Cognitive Computing System (CCS) that consists of humans and software that work together as a 'Distributed Federated Brain'. Motivated by neuromorphic processing models, we present an approach that uses hyperdimensional symbolic semantic vector representations of the services/devices and workflows. We show how these can be used to perform decentralized service/device discovery and workflow composition in the context of a dynamic communications re-planning scenario. In this paper, we describe how emerging analogue AI 'In Memory' and 'Near Memory' computing can be used to efficiently perform some of the required hyperdimensional vector computation (HDC). We present an evaluation of the performance of an energy-efficient phase change memory device (PCM) that can perform the required vector operations and discuss how such devices could be used in energy-critical 'edge of network' tactical MDO operations.

show abstract

Section: Representing Radio's As Vsa Vectorsmentioning

confidence: 99%

Energy efficient 'in memory' computing to enable decentralised service workflow composition in support of multi-domain operations

Bent

Simpkin

Taylor

et al. 2021

Artificial Intelligence and Machine Learning for Multi-Domain Operations Applications III

View full text Add to dashboard Cite

show abstract

“…We note, however, that distribution representations include not only random projection based methods [15], [16], [17], [18], [43], [44], [45] but also a number of other representation schemes for vectors, such as those based on receptive fields of other types [46] or compositional methods [47], [48], [49], [50]. DRs can be used to represent certain types of images using a special type of LiRA receptive fields [51], [52], [53].…”

Section: Discussionmentioning

confidence: 93%

“…We investigated [18] the estimates of similarity measures for real-valued vectors by binary codevectors obtained by projecting with a random binary matrix and then using the output threshold transform that allows us to adjust the degree of sparsity (the fraction of non-zero components) of binary codevectors. The similarity of binary codevectors was estimated by measures based on dot product (normalized to the codevector dimension).…”

Section: Similarity Estimation With Distributed Representations Of Vementioning

confidence: 99%

Neural Distributed Representations of Vector Data in Intelligent Information Technologies

Gritsenko¹,

Revunova²,

Rachkovskij³

2018

Kibern. vyčisl. teh.

View full text Add to dashboard Cite

Introduction. Distributed representation (DR) of data is a form of a vector representation,where each object is represented by a set of vector components, and each vector component can belong to representations of many objects. In ordinary vector representations, the meaning of each component is defined, which cannot be said about DR. However, the similarity of RP vectors reflects the similarity of the objects they represent.DR is a neural network approach based on modeling the representation of information in the brain, resulted from ideas about a "distributed" or "holographic" representations. DRs have a large information capacity, allow the use of a rich arsenal of methods developed for vector data, scale well for processing large amounts of data, and have a number of other advantages. Methods for data transformation to DRs have been developed for data of various types -from scalar and vector to graphs.The purpose of the article is to provide an overview of a part of the work of the Department of Neural Information Processing Technologies (International Center) in the field of neural network distributed representations. The approach is a development of the ideas of Nikolai Mikhailovich Amosov and his scientific school of modeling the structure and functions of the brain.Scope. The formation of distributed representations from the original vector representations of objects using random projection is considered. With the help of the DR, it is possible to efficiently estimate the similarity of the original objects represented by numerical vectors. Gritsenko V.I., Rachkovskij D.A., Revunova E.G. ISSN 2519-2205 (Online), ISSN 0454-9910 (Print). Киб. и выч. техн. 2018. № 4 (194) 8The use of DR allows developing regularization methods for obtaining a stable solution of discrete ill-posed inverse problems, increasing the computational efficiency and accuracy of their solution, analyzing analytically the accuracy of the solution. Thus DRs allow for increasing the efficiency of information technologies applying them.Conclusions. DRs of various data types can be used to improve the efficiency and intelligence level of information technologies. DRs have been developed for both weakly structured data, such as vectors, and for complex structured representations of objects, such as sequences, graphs of knowledge-base situations (episodes), etc. Transformation of different types of data into the DR vector format allows unifying the basic information technologies of their processing and achieving good scalability with an increase in the amount of data processed.In future, distributed representations will naturally combine information on structure and semantics to create computationally efficient and qualitatively new information technologies in which the processing of relational structures from knowledge bases is performed by the similarity of their DRs. The neurobiological relevance of distributed representations opens up the possibility of creating intelligent information technologies based on them that function similarly to...

show abstract

“…Note that such data representation schemes by similarity preserving binary vectors have been developed for objects represented by various data types, mainly for (feature) vectors (see survey in [131]), but also for structured data types such as sequences [102,72,85,86] and graphs [127,128,148,136,62,134]. A significant part of this research is developed in the framework of distributed representations [45,76,106,126,89], including binary sparse distributed representations [102,98,103,127,128,113,114,137,138,139,148,135,136,61,134,129,130,131,132,31,33] and dense distributed representations [75,76] (see [82,84,87,88,83] for examples of their applications).…”

Section: Generalization In Namsmentioning

confidence: 99%

Neural Autoassociative Memories for Binary Vectors: A Survey

Gritsenko¹,

Rachkovskij²,

Frolov³

et al. 2017

Kibern. vyčisl. teh.

View full text Add to dashboard Cite

Introduction. Neural network models of autoassociative, distributed memory allow storage and retrieval of many items (vectors) where the number of stored items can exceed the vector dimension (the number of neurons in the network). This opens the possibility of a sublinear time search (in the number of stored items) for approximate nearest neighbors among vectors of high dimension.The purpose of this paper is to review models of autoassociative, distributed memory that can be naturally implemented by neural networks (mainly with local learning rules and iterative dynamics based on information locally available to neurons).Scope. The survey is focused mainly on the networks of Hopfield, Willshaw and Potts, that have connections between pairs of neurons and operate on sparse binary vectors. We discuss not only autoassociative memory, but also the generalization properties of these networks. We also consider neural networks with higher-order connections and networks with a bipartite graph structure for non-binary data with linear constraints.Conclusions. In conclusion we discuss the relations to similarity search, advantages and drawbacks of these techniques, and topics for further research. An interesting and still not completely resolved question is whether neural autoassociative memories can search for approximate nearest neighbors faster than other index structures for similarity search, in particular for the case of very high dimensional vectors. Introduction. Neural network models of autoassociative, distributed memory allow storage and retrieval of many items (vectors) where the number of stored items can exceed the vector dimension (the number of neurons in the network). This opens the possibility of a sublinear time search (in the number of stored items) for approximate nearest neighbors among vectors of high dimension.The purpose of this paper is to review models of autoassociative, distributed memory that can be naturally implemented by neural networks (mainly with local learning rules and iterative dynamics based on information locally available to neurons).Scope. The survey is focused mainly on the networks of Hopfield, Willshaw and Potts, that have connections between pairs of neurons and operate on sparse binary vectors. We (Online), ISSN 0454-9910 (Print). Киб. и выч. техн. 2017. № 2 (188) 6 discuss not only autoassociative memory, but also the generalization properties of these networks. We also consider neural networks with higher-order connections and networks with a bipartite graph structure for non-binary data with linear constraints.Conclusions. In conclusion we discuss the relations to similarity search, advantages and drawbacks of these techniques, and topics for further research. An interesting and still not completely resolved question is whether neural autoassociative memories can search for approximate nearest neighbors faster than other index structures for similarity search, in particular for the case of very high dimensional vectors. ISSN 2519-2205 (Online), ISSN 0454-9910 (Print). Киб...

show abstract

Estimation of Vectors Similarity by Their Randomized Binary Projections

Cited by 11 publications

References 45 publications

Energy efficient 'in memory' computing to enable decentralised service workflow composition in support of multi-domain operations

Energy efficient 'in memory' computing to enable decentralised service workflow composition in support of multi-domain operations

Neural Distributed Representations of Vector Data in Intelligent Information Technologies

Neural Autoassociative Memories for Binary Vectors: A Survey

Contact Info

Product

Resources

About