Bringing the Blessing of Dimensionality to the Edge

Tyukin, Ivan; Gorban, Alexander N.; McEwan, Alistair A.; Meshkinfamfard, Sepehr

doi:10.1109/iciai.2019.8850825

Cited by 4 publications

(7 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The high risk/normal situation classifier is prepared by supervised training on situations with diagnosed errors (universal construction). The online training algorithm could be very simple like Fisher's linear discriminants or their ensembles [30,39,40,43,44]. Correction rules for high-risk situations are specific to a particular problem.…”

Section: Internal Signals Inputs Outputsmentioning

confidence: 99%

“…The correction methods were tested on various AI applications for videostream processing: detection of faces for security applications and detection of pedestrians [39,44,51], translation of Sign Language into text for communication between deaf-mute people [52], knowledge transfer between AI systems [53], medical image analysis, scanning and classifying archaeological artifacts [54], etc., and even to some industrial systems with relatively high level of errors [43].…”

Section: Some Applicationsmentioning

confidence: 99%

“…If γ 1 then the probability to find a data point in the intersection of the balls (the 'area of confusion' between clusters) is negligible for uniform distributions in balls, isotropic Gaussian distributions and always when small volume implies small probability. Therefore, the clustering of mistakes for correction of high-dimensional machine learning systems gives good results even if clusters are not very good in the standard measures, and correction of clustered mistakes requires much fewer correctors for the same or even better accuracy [43].…”

Section: Clustering In High Dimensionsmentioning

confidence: 99%

“…We implemented the correctors with separation of clustered false-positive mistakes from the set of true positive and tested them on the classical face detection task [43]. The legacy object detector was an OpenCV implementation of the Haar face detector.…”

Section: Clustering In High Dimensionsmentioning

confidence: 99%

“…Further increase of dimension (the number of principal components retained) can even damage the performance because the number of features does not coincide with the dimension of the dataset, and the whitening with retained minor component can lead to ill-posed problems and loss of stability. For more detail, we refer to [43].…”

Section: Clustering In High Dimensionsmentioning

confidence: 99%

See 4 more Smart Citations

High-Dimensional Brain in a High-Dimensional World: Blessing of Dimensionality

Gorban

Makarov

Tyukin

2020

Entropy

Self Cite

View full text Add to dashboard Cite

High-dimensional data and high-dimensional representations of reality are inherent features of modern Artificial Intelligence systems and applications of machine learning. The well-known phenomenon of the "curse of dimensionality" states: many problems become exponentially difficult in high dimensions. Recently, the other side of the coin, the "blessing of dimensionality", has attracted much attention. It turns out that generic high-dimensional datasets exhibit fairly simple geometric properties. Thus, there is a fundamental tradeoff between complexity and simplicity in high dimensional spaces. Here we present a brief explanatory review of recent ideas, results and hypotheses about the blessing of dimensionality and related simplifying effects relevant to machine learning and neuroscience. of 18programming was considered a method of dimensionality reduction in the optimization of a multi-stage decision process. Bellman returned to the problem of dimensionality reduction many times in different contexts [3]. Now, dimensionality reduction is an essential element of the engineering (the "practical man") approach to mathematical modeling [4]. Many model reduction methods were developed and successfully implemented in applications, from various versions of principal component analysis to approximation by manifolds, graphs, and complexes [5][6][7], and low-rank tensor network decompositions [8,9].Various reasons and forms of the curse of dimensionality were classified and studied, from the obvious combinatorial explosion (for example, for n binary Boolean attributes, to check all the combinations of values we have to analyze 2 n cases) to more sophisticated distance concentration: in a high-dimensional space, the distances between randomly selected points tend to concentrate near their mean value, and the neighbor-based methods of data analysis become useless in their standard forms [10,11]. Many "good" polynomial time algorithms become useless in high dimensions.Surprisingly, however, and despite the expected challenges and difficulties, common-sense heuristics based on the simple and the most straightforward methods "can yield results which are almost surely optimal" for high-dimensional problems [12]. Following this observation, the term "blessing of dimensionality" was introduced [12,13]. It was clearly articulated as a basis of future data mining in the Donoho "Millenium manifesto" [14]. After that, the effects of the blessing of dimensionality were discovered in many applications, for example in face recognition [15], in analysis and separation of mixed data that lie on a union of multiple subspaces from their corrupted observations [16], in multidimensional cluster analysis [17], in learning large Gaussian mixtures [18], in correction of errors of multidimensonal machine learning systems [19], in evaluation of statistical parameters [20], and in the development of generalized principal component analysis that provides low-rank estimates of the natural parameters by projecting the saturated model parameters [21]....

show abstract

Section: Internal Signals Inputs Outputsmentioning

confidence: 99%

Section: Some Applicationsmentioning

confidence: 99%

Section: Clustering In High Dimensionsmentioning

confidence: 99%

Section: Clustering In High Dimensionsmentioning

confidence: 99%

Section: Clustering In High Dimensionsmentioning

confidence: 99%

See 3 more Smart Citations

High-Dimensional Brain in a High-Dimensional World: Blessing of Dimensionality

Gorban

Makarov

Tyukin

2020

Entropy

Self Cite

View full text Add to dashboard Cite

show abstract

Local intrinsic dimensionality estimators based on concentration of measure

Bac¹,

Zinovyev²

2020

2020 International Joint Conference on Neural Networks (IJCNN)

View full text Add to dashboard Cite

Local intrinsic dimensionality estimators based on concentration of measure

Bac¹,

Zinovyev²

2020

Preprint

View full text Add to dashboard Cite

Intrinsic dimensionality (ID) is one of the most fundamental characteristics of multi-dimensional data point clouds. Knowing ID is crucial to choose the appropriate machine learning approach as well as to understand its behavior and validate it. ID can be computed globally for the whole data distribution, or computed locally in different regions of the dataset. In this paper, we introduce new local estimators of ID based on linear separability of multi-dimensional data point clouds, which is one of the manifestations of concentration of measure. We empirically study the properties of these estimators and compare them with other recently introduced ID estimators exploiting various effects of measure concentration. Observed differences between estimators can be used to anticipate their behaviour in practical applications.

show abstract

Bringing the Blessing of Dimensionality to the Edge

Cited by 4 publications

References 31 publications

High-Dimensional Brain in a High-Dimensional World: Blessing of Dimensionality

High-Dimensional Brain in a High-Dimensional World: Blessing of Dimensionality

Local intrinsic dimensionality estimators based on concentration of measure

Local intrinsic dimensionality estimators based on concentration of measure

Contact Info

Product

Resources

About