LGTM: A Fast and Accurate kNN Search Algorithm in High-Dimensional Spaces

Arai, Yusuke; Amagata, Daichi; Fujita, Sumio; Hara, Takahiro

doi:10.1007/978-3-030-86475-0_22

Cited by 2 publications

(1 citation statement)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Optimizing Entry Point If we have multiple entry point candidates, starting with the one closest to the query dramatically speeds up the search [3,6]. We propose a novel and straightforward entry point selection method utilizing kmeans clustering.…”

Section: Database Subsamplingmentioning

confidence: 99%

General and Practical Tuning Method for Off-the-Shelf Graph-Based Index: SISAP Indexing Challenge Report by Team UTokyo

Oguri,

Matsui

2023

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Despite the efficacy of graph-based algorithms for Approximate Nearest Neighbor (ANN) searches, the optimal tuning of such systems remains unclear. This study introduces a method to tune the performance of off-the-shelf graph-based indexes, focusing on the dimension of vectors, database size, and entry points of graph traversal. We utilize a black-box optimization algorithm to perform integrated tuning to meet the required levels of recall and Queries Per Second (QPS). We applied our approach to Task A of the SISAP 2023 Indexing Challenge and got second place in the 10M and 30M tracks. It improves performance substantially compared to brute force methods. This research offers a universally applicable tuning method for graph-based indexes, extending beyond the specific conditions of the competition to broader uses.

show abstract

Section: Database Subsamplingmentioning

confidence: 99%

General and Practical Tuning Method for Off-the-Shelf Graph-Based Index: SISAP Indexing Challenge Report by Team UTokyo

Oguri,

Matsui

2023

Lecture Notes in Computer Science

View full text Add to dashboard Cite

show abstract

Scalable and Accurate Density-Peaks Clustering on Fully Dynamic Data

Amagata

2022

2022 IEEE International Conference on Big Data (Big Data)

Self Cite

View full text Add to dashboard Cite

Clustering is a primitive and important operator that analyzes a given dataset to discover its hidden patterns and features. Because datasets are usually updated dynamically (i.e., it accepts continuous insertions and arbitrary deletions), analyzing such dynamic data is also an important topic, and dynamic clustering effectively supports it, but is a challenging problem. In this paper, we consider the problem of densitypeaks clustering (DPC) on dynamic data. DPC is one of the density-based clustering algorithms and attracts attention for many applications, due to its effectiveness. We investigate the hardness of this problem theoretically to measure the efficiencies of dynamic DPC algorithms. We prove that any exact solutions are costly, and propose an approximation algorithm to enable faster updates. We conduct experiments on real datasets, and the results confirm that our algorithm is much faster and more accurate than state-of-the-art.

show abstract

LGTM: A Fast and Accurate kNN Search Algorithm in High-Dimensional Spaces

Cited by 2 publications

References 8 publications

General and Practical Tuning Method for Off-the-Shelf Graph-Based Index: SISAP Indexing Challenge Report by Team UTokyo

General and Practical Tuning Method for Off-the-Shelf Graph-Based Index: SISAP Indexing Challenge Report by Team UTokyo

Scalable and Accurate Density-Peaks Clustering on Fully Dynamic Data

Contact Info

Product

Resources

About