A Comprehensive Survey and Experimental Comparison of Graph-Based Approximate Nearest Neighbor Search

Wang, Mengzhao; Xu, Xinhua; Ye, Qiang; Wang, Yuxiang

doi:10.48550/arxiv.2101.12631

Cited by 8 publications

(13 citation statements)

References 63 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The exact k-nearest neighbor problem asks for exact (true) k-nearest neighbors of every query point q. Another probabilistic version of the k-nearest neighbor search [44] , [61] The approximate version [2], [54], [1], [86] for every query point q ∈ Q, looks for its approximate neighbor r ∈ R satisfying d (q, r ) ≤ (1 + )d (q, NN(q)), where > 0 is fixed and NN(q) is the exact first nearest neighbor of q.…”

Section: The K-nearest Neighbor Search and Overview Of Resultsmentioning

confidence: 99%

A new compressed cover tree for k-nearest neighbour search and the stable-under-noise mergegram of a point cloud

Yury¹

2022

Preprint

View full text Add to dashboard Cite

This thesis consists of two topics related to computational geometry and one topic related to topological data analysis (TDA), which combines fields of computational geometry and algebraic topology for analyzing data.The first part studies the classical problem of finding k nearest neighbors to m query points in a larger set of n reference points in any metric space. The well-known work by Beygelzimer, Kakade, and Langford in ICML 2006 introduced cover trees and claimed to guarantee a nearlinear time complexity in the number n of reference points. Section 5.3 of Curtin's PhD (2015) pointed out that the proof of this result was wrong. The key step of the original proof attempted to show that the number of iterations can be estimated by multiplying the length of the longest rootto-leaf path in a cover tree by a constant factor. However, this estimate can miss many potential nodes in several branches of a cover tree, that should be considered during the search. The same erroneous argument was unfortunately repeated in several subsequent papers. This chapter gives formal counterexamples to the time complexity estimates of the cover tree construction algorithm and the nearest neighbors search in the past work. We prove correct analogs of the past claims with slightly better bounds for any number k ≥ 1 of neighbors. A new compressed cover tree guarantees a parameterized time complexity that is near-linear in the maximum size of both query and reference set.The second part is about the construction of a Minimum Spanning Tree (MST) on any finite metric space. This is an efficient way to visualize any unstructured data given only by distances, for example by any metric graph connecting abstract data points. Though many efficient algorithms for MST were developed, there was only one past attempt to justify a nearlinear time complexity in the size of a given metric space. In 2010 March, Ram, and Gray claimed that MST of any finite metric space can be constructed in a parametrized near-linear time. However, in this work we have discovered a counterexample showing that the provided proof was incorrect. The past obstacles are now resolved by the new concepts of a compressed cover tree and a minimized expansion constant. The main contribution is the first MST algorithm that correctly justifies a parametrized a near-linear time complexity.The third part extends the key concept of persistence within Topological Data Analysis in a new direction. TDA quantifies topological shapes hidden in unorganized data such as clouds of unordered points. In the 0-dimensional case, the distance-based persistence is determined by a i First of all, I would like to thank my primary supervisor, Dr. Vitaliy Kurlin, for giving me the opportunity to pursue a Ph.D. in a field about which I am very passionate and for always supporting me during these years. His incredible experience has been inspiring to me. I am grateful for all our fruitful conversations that helped me progress in my research.I would also like to thank my second supervisor, Dr. Marja K...

show abstract

Section: The K-nearest Neighbor Search and Overview Of Resultsmentioning

confidence: 99%

A new compressed cover tree for k-nearest neighbour search and the stable-under-noise mergegram of a point cloud

Yury¹

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Graphs can be used as effective indexes to accelerate nearest neighbors search [55,15]. Existing graph construction methods generally propose different rules to generate graphs, which cannot provide adaptivity for different workloads [102]. Baranchuk et al [5] employ DRL to optimize the graph for nearest neighbors search.…”

Section: Index Structure Constructionmentioning

confidence: 99%

A Survey on Deep Reinforcement Learning for Data Processing and Analytics

Cai¹,

Cui²,

Xiong³

et al. 2021

Preprint

View full text Add to dashboard Cite

Data processing and analytics are fundamental and pervasive. Algorithms play a vital role in data processing and analytics where many algorithm designs have incorporated heuristics and general rules from human knowledge and experience to improve their effectiveness. Recently, reinforcement learning, deep reinforcement learning (DRL) in particular, is increasingly explored and exploited in many areas because it can learn better strategies in complicated environments it is interacting with than statically designed algorithms. Motivated by this trend, we provide a comprehensive review of recent works focusing on utilizing deep reinforcement learning to improve data processing and analytics. First, we present an introduction to key concepts, theories, and methods in deep reinforcement learning. Next, we discuss deep reinforcement learning deployment on database systems, facilitating data processing and analytics in various aspects, including data organization, scheduling, tuning, and indexing. Then, we survey the application of deep reinforcement learning in data processing and analytics , ranging from data preparation, natural language interface to healthcare, fintech, etc. Finally, we discuss important open challenges and future research directions of using deep reinforcement learning in data processing and analytics.

show abstract

“…At the other end of the spectrum of ANNS indices are graph-based indexing algorithms [28,33,34,43,51,52]. Several comparative studies [7,25,41,58] of ANNS algorithms have concluded that they significantly out-perform other techniques in terms of search throughput on a range of realworld static datasets. These algorithms are also widely used in the industry at scale.…”

Section: Shortcoming Of Existing Algorithmsmentioning

confidence: 99%

“…Even though this abstraction of ANN search is widely studied, it does not capture many important real-world scenarios where user interactions with a system creates and destroys data, and results in updates to 𝑃 (especially in the literature on graph-based ANNS indices [58]). For example, consider an enterprise-search scenario where the system indexes sentences in documents generated by users across an enterprise.…”

Section: Introductionmentioning

confidence: 99%

FreshDiskANN: A Fast and Accurate Graph-Based ANN Index for Streaming Similarity Search

Singh,

Subramanya,

Krishnaswamy

et al. 2021

Preprint

View full text Add to dashboard Cite

Approximate nearest neighbor search (ANNS) is a fundamental building block in information retrieval with graphbased indices being the current state-of-the-art [7] and widely used in the industry. Recent advances [51] in graph-based indices have made it possible to index and search billion-point datasets with high recall and millisecond-level latency on a single commodity machine with an SSD.However, existing graph algorithms for ANNS support only static indices that cannot reflect real-time changes to the corpus required by many key real-world scenarios (e.g. index of sentences in documents, email or a news index). To overcome this drawback, the current industry practice for manifesting updates into such indices is to periodically re-build these indices, which can be prohibitively expensive.In this paper, we present the first graph-based ANNS index that reflects corpus updates into the index in real-time without compromising on search performance. Using update rules for this index, we design FreshDiskANN, a system that can index over a billion points on a workstation with an SSD and limited memory, and support thousands of concurrent real-time inserts, deletes and searches per second each, while retaining > 95% 5-recall@5. This represents a 5-10x reduction in the cost of maintaining freshness in indices when compared to existing methods.

show abstract

A Comprehensive Survey and Experimental Comparison of Graph-Based Approximate Nearest Neighbor Search

Cited by 8 publications

References 63 publications

A new compressed cover tree for k-nearest neighbour search and the stable-under-noise mergegram of a point cloud

A new compressed cover tree for k-nearest neighbour search and the stable-under-noise mergegram of a point cloud

A Survey on Deep Reinforcement Learning for Data Processing and Analytics

FreshDiskANN: A Fast and Accurate Graph-Based ANN Index for Streaming Similarity Search

Contact Info

Product

Resources

About