“…Later other researchers began to explore the regularizing properties of random projection, for example, for classification problems and machine learning [20], and, more recently, for solving inverse problems [21]. Since the approach of random projection, along with improving the accuracy of the solution by regularization, reduces the computational complexity of the solution, we have managed to develop algorithms that provide an accurate and fast solution for discrete inverse problems [22], [23], [24], [25], [26], [27], [28].…”
Section: Distributed Representations Based On Random Projections For mentioning
confidence: 99%
“…Search for the optimal number of rows of a random matrix. In [23], expressions for the recovery error of x were obtained for the random projection method: Fig. 3.…”
Section: Distributed Representations Based On Random Projections For mentioning
Introduction. Distributed representation (DR) of data is a form of a vector representation,where each object is represented by a set of vector components, and each vector component can belong to representations of many objects. In ordinary vector representations, the meaning of each component is defined, which cannot be said about DR. However, the similarity of RP vectors reflects the similarity of the objects they represent.DR is a neural network approach based on modeling the representation of information in the brain, resulted from ideas about a "distributed" or "holographic" representations. DRs have a large information capacity, allow the use of a rich arsenal of methods developed for vector data, scale well for processing large amounts of data, and have a number of other advantages. Methods for data transformation to DRs have been developed for data of various types -from scalar and vector to graphs.The purpose of the article is to provide an overview of a part of the work of the Department of Neural Information Processing Technologies (International Center) in the field of neural network distributed representations. The approach is a development of the ideas of Nikolai Mikhailovich Amosov and his scientific school of modeling the structure and functions of the brain.Scope. The formation of distributed representations from the original vector representations of objects using random projection is considered. With the help of the DR, it is possible to efficiently estimate the similarity of the original objects represented by numerical vectors. Gritsenko V.I., Rachkovskij D.A., Revunova E.G. ISSN 2519-2205 (Online), ISSN 0454-9910 (Print). Киб. и выч. техн. 2018. № 4 (194)
8The use of DR allows developing regularization methods for obtaining a stable solution of discrete ill-posed inverse problems, increasing the computational efficiency and accuracy of their solution, analyzing analytically the accuracy of the solution. Thus DRs allow for increasing the efficiency of information technologies applying them.Conclusions. DRs of various data types can be used to improve the efficiency and intelligence level of information technologies. DRs have been developed for both weakly structured data, such as vectors, and for complex structured representations of objects, such as sequences, graphs of knowledge-base situations (episodes), etc. Transformation of different types of data into the DR vector format allows unifying the basic information technologies of their processing and achieving good scalability with an increase in the amount of data processed.In future, distributed representations will naturally combine information on structure and semantics to create computationally efficient and qualitatively new information technologies in which the processing of relational structures from knowledge bases is performed by the similarity of their DRs. The neurobiological relevance of distributed representations opens up the possibility of creating intelligent information technologies based on them that function similarly to...
“…Later other researchers began to explore the regularizing properties of random projection, for example, for classification problems and machine learning [20], and, more recently, for solving inverse problems [21]. Since the approach of random projection, along with improving the accuracy of the solution by regularization, reduces the computational complexity of the solution, we have managed to develop algorithms that provide an accurate and fast solution for discrete inverse problems [22], [23], [24], [25], [26], [27], [28].…”
Section: Distributed Representations Based On Random Projections For mentioning
confidence: 99%
“…Search for the optimal number of rows of a random matrix. In [23], expressions for the recovery error of x were obtained for the random projection method: Fig. 3.…”
Section: Distributed Representations Based On Random Projections For mentioning
Introduction. Distributed representation (DR) of data is a form of a vector representation,where each object is represented by a set of vector components, and each vector component can belong to representations of many objects. In ordinary vector representations, the meaning of each component is defined, which cannot be said about DR. However, the similarity of RP vectors reflects the similarity of the objects they represent.DR is a neural network approach based on modeling the representation of information in the brain, resulted from ideas about a "distributed" or "holographic" representations. DRs have a large information capacity, allow the use of a rich arsenal of methods developed for vector data, scale well for processing large amounts of data, and have a number of other advantages. Methods for data transformation to DRs have been developed for data of various types -from scalar and vector to graphs.The purpose of the article is to provide an overview of a part of the work of the Department of Neural Information Processing Technologies (International Center) in the field of neural network distributed representations. The approach is a development of the ideas of Nikolai Mikhailovich Amosov and his scientific school of modeling the structure and functions of the brain.Scope. The formation of distributed representations from the original vector representations of objects using random projection is considered. With the help of the DR, it is possible to efficiently estimate the similarity of the original objects represented by numerical vectors. Gritsenko V.I., Rachkovskij D.A., Revunova E.G. ISSN 2519-2205 (Online), ISSN 0454-9910 (Print). Киб. и выч. техн. 2018. № 4 (194)
8The use of DR allows developing regularization methods for obtaining a stable solution of discrete ill-posed inverse problems, increasing the computational efficiency and accuracy of their solution, analyzing analytically the accuracy of the solution. Thus DRs allow for increasing the efficiency of information technologies applying them.Conclusions. DRs of various data types can be used to improve the efficiency and intelligence level of information technologies. DRs have been developed for both weakly structured data, such as vectors, and for complex structured representations of objects, such as sequences, graphs of knowledge-base situations (episodes), etc. Transformation of different types of data into the DR vector format allows unifying the basic information technologies of their processing and achieving good scalability with an increase in the amount of data processed.In future, distributed representations will naturally combine information on structure and semantics to create computationally efficient and qualitatively new information technologies in which the processing of relational structures from knowledge bases is performed by the similarity of their DRs. The neurobiological relevance of distributed representations opens up the possibility of creating intelligent information technologies based on them that function similarly to...
“…Recently, different kinds of randomized algorithms have been proposed to compute the lowrank matrix approximation [1,5,7,12,14,27,28,29,32,33,35,41,42]. The main idea is to obtain a projection by a random matrix (Gaussian matrix or matrix generated by the sub-sampled randomized Fourier transform (SRFT) [29,33,42]) or random sampling [1,28] with preconditioning [5,32]; refer also to the review paper [14]. Gu presented a randomized algorithm within the subspace iteration framework which gives accurate low-rank approximations of high probability, for matrices with rapidly decaying singular values [13].…”
In this paper, we apply randomized algorithms to approximate the total least squares (TLS) solution of the problem Ax ≈ b in the large-scale discrete ill-posed problems. A regularization technique, based on the multiplicative randomization and the subspace iteration, is proposed to obtain the approximate core problem. In the error analysis, we provide upper bounds for the errors of the solution and the residual of the randomized core reduction.Illustrative numerical examples and comparisons are presented.
We propose a transformation of real input vectors to output binary vectors by projection using a binary random matrix with elements {0,1} and thresholding. We investigate the rate of convergence of the distribution of vector components before binarization to the Gaussian distribution as well as its relationship to the estimation error of the angle between the input vectors by the binarized output vectors. It is shown that for the choice of projection parameters that provide nearly-Gaussian distribution, the experimental and analytical errors are close.Keywords: binary random projections, convergence to the Gaussian distribution, estimate of the similarity of vectors.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.