A Survey on Locality Sensitive Hashing Algorithms and their Applications

Jafari, Omid; Maurya, Preeti; Nagarkar, Parth; Islam, Khandker Mushfiqul; Crushev, Chidambaram

doi:10.48550/arxiv.2102.08942

Cited by 3 publications

(3 citation statements)

References 71 publications

(105 reference statements)

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…The main idea behind LSH is that the points that are close in the original space will map to the same hash buckets in the low-dimensional spaces with a higher probability compared to the points that are far from each other in the original space. Since LSH is used in many different applications [8], there have been several works that have been proposed to improve its accuracy and performance [16,5,7,3,10,12,13,9,15].…”

Section: Locality Sensitive Hashingmentioning

confidence: 99%

Experimental Analysis of Machine Learning Techniques for Finding Search Radius in Locality Sensitive Hashing

Jafari¹,

Nagarkar²

2022

Preprint

View full text Add to dashboard Cite

Finding similar data in high-dimensional spaces is one of the important tasks in multimedia applications. Approaches introduced to find exact searching techniques often use tree-based index structures which are known to suffer from the curse of the dimensionality problem that limits their performance. Approximate searching techniques prefer performance over accuracy and they return good enough results while achieving a better performance. Locality Sensitive Hashing (LSH) is one of the most popular approximate nearest neighbor search techniques for high-dimensional spaces. One of the most time-consuming processes in LSH is to find the neighboring points in the projected spaces. An improved LSH-based index structure, called radius-optimized Locality Sensitive Hashing (roLSH ) has been proposed to utilize Machine Learning and efficiently find these neighboring points; thus, further improve the overall performance of LSH. In this paper, we extend roLSH by experimentally studying the effect of different types of famous Machine Learning techniques on overall performance. We compare ten regression techniques on four real-world datasets and show that Neural Networkbased techniques are the best fit to be used in roLSH as their accuracy and performance trade-off are the best compared to the other techniques.

show abstract

Section: Locality Sensitive Hashingmentioning

confidence: 99%

Experimental Analysis of Machine Learning Techniques for Finding Search Radius in Locality Sensitive Hashing

Jafari¹,

Nagarkar²

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Através de func ¸ões hash aleatórias, o Hash Sensível à Localidade (do inglês, Locality Sensitive Hashing, ou LSH) é capaz de mapear dados de alta dimensão para uma dimensão inferior, tornando-se uma das soluc ¸ões mais populares para encontrar o ANN [9]. Inicialmente proposto por Indyk e Motwani [8], seu uso tem aplicac ¸ões em diversas áreas, como machine learning, ciências geológicas, compressão de dados, e investigac ¸ão forense [8], [9], [18].…”

Section: A Hash Sensível à Localidadeunclassified

Auditoria de aplicações de Big Data usando Hashes de Similaridade e Blockchains

Oliveira

Assumpção

Ortíz

et al. 2022

Anais Estendidos Do XII Simpósio Brasileiro De Engenharia De Sistemas Computacionais (SBESC Estendido 2022)

View full text Add to dashboard Cite

Com a expansão das aplicações de BigData, garantir a segurança e confiabilidade dos dados armazenados tornou-se uma tarefa desafiadora. Tal desafio é especialmente preocupante quando se considera o monitoramento de infraestruturas críticas, em especial aquelas que contemplam ativos físicos monitorados por sensores e dispositivos IoT de coleta de dados. Uma alternativa consiste no uso de blockchains como mecanismo de auditoria de aplicações Big Data a partir da técnica off-chain, onde os pacotes de dados brutos são armazenados em um sistema de banco de dados convencional e apenas um resumo criptográfico dos dados é escrito no blockchain. Embora bastante aplicada na literatura recente sobre o tema, essa estratégia não permite a auditoria de dados em cenários de perda parcial de informações, onde pacotes de dados correspondentes a subconjuntos do pacote original precisam ser verificados. Este artigo propõe uma estratégia de auditoria de dados em aplicações Big Data que emprega hashes de similaridade para estender as funcionalidades do modelo off-chain. Usado em conjunto com resumos criptográficos e smart contracts, tais hashes permitem auditar pacotes de dados distintos em situações de possível perda parcial, possibilitando diferenciar uma situação não intencional de uma tentativa deliberada de fraude. Em nossos experimentos, avaliamos os algoritmos Minhash e Simhash, apontando resultados computacionais que indicam que o Minhash é bastante promissor para esse tipo de aplicação, e pode contribuir significativamente para a robustez de processos de auditoria em aplicações Big Data.

show abstract

“…The aim of similarity learning is to learn a siamese network that measures how similar or associative two objects are. Generally, there are four common types of similarity and metric distance learning, including classification [61,30,52], regression [28,47], ranking [6,62] and locality sensitive hashing [8,25]. In this work, we focus on classification similarity learning, i.e., learning a similarity function to classify other objects given the class of one object.…”

Section: Ablation Studymentioning

confidence: 99%

Learning by Examples Based on Multi-level Optimization

Mo¹,

Xie²

2021

Preprint

View full text Add to dashboard Cite

Learning by examples, which learns to solve a new problem by looking into how similar problems are solved, is an effective learning method in human learning. When a student learns a new topic, he/she finds out exemplar topics that are similar to this new topic and studies the exemplar topics to deepen the understanding of the new topic. We aim to investigate whether this powerful learning skill can be borrowed from humans to improve machine learning as well. In this work, we propose a novel learning approach called Learning By Examples (LBE). Our approach automatically retrieves a set of training examples that are similar to query examples and predicts labels for query examples by using class labels of the retrieved examples. We propose a three-level optimization framework to formulate LBE which involves three stages of learning: learning a Siamese network to retrieve similar examples; learning a matching network to make predictions on query examples by leveraging class labels of retrieved similar examples; learning the "ground-truth" similarities between training examples by minimizing the validation loss. We develop an efficient algorithm to solve the LBE problem and conduct extensive experiments on various benchmarks where the results demonstrate the effectiveness of our method on both supervised and few-shot learning.Preprint. Under review.

show abstract

A Survey on Locality Sensitive Hashing Algorithms and their Applications

Cited by 3 publications

References 71 publications

Experimental Analysis of Machine Learning Techniques for Finding Search Radius in Locality Sensitive Hashing

Experimental Analysis of Machine Learning Techniques for Finding Search Radius in Locality Sensitive Hashing

Auditoria de aplicações de Big Data usando Hashes de Similaridade e Blockchains

Learning by Examples Based on Multi-level Optimization

Contact Info

Product

Resources

About