Dynamic hashing schemes

Enbody, Richard; Du, H. C.

doi:10.1145/46157.330532

Cited by 101 publications

(25 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Also, the usage of a page table to preserve mappings between sections (pages) of the table and their locations (page servers) makes DPH a directory based [15] approach.…”

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

Distributed Paged Hash Tables

Rufino

Pina

Alves

et al. 2003

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Abstract. In this paper we present the design and implementation of DPH, a storage layer for cluster environments. DPH is a Distributed Data Structure (DDS) based on the distribution of a paged hash table. It combines main memory with file system resources across the cluster in order to implement a distributed dictionary that can be used for the storage of very large data sets with key based addressing techniques. The DPH storage layer is supported by a collection of cluster-aware utilities and services. Access to the DPH interface is provided by a user-level API. A preliminary performance evaluation shows promising results.

show abstract

“…Also, the usage of a page table to preserve mappings between sections (pages) of the table and their locations (page servers) makes DPH a directory based [15] approach.…”

Section: Discussionmentioning

confidence: 99%

“…Static hashing may not also be able to guarantee O(1) retrieval times when buckets overflow. To counterwork these limitations several dynamic hashing [15] techniques have been proposed, such as Linear Hashing (LH) [16] and Extendible Hashing (EH) [17], along with some variants.…”

Section: Related Workmentioning

confidence: 99%

Distributed Paged Hash Tables

Rufino

Pina

Alves

et al. 2003

Lecture Notes in Computer Science

View full text Add to dashboard Cite

show abstract

“…To generate a hash signature for a term (word) we can use various hashing schemes whose goal is that very different documents should have very different signatures [8,19]. We use the sdbm hashing function in our implementation and experiments, which is the most popular function for simhash implementations, and is also used in other popular text handling systems like Berkeley DB due to its effectiveness.…”

Section: ) Term Hash Functionmentioning

confidence: 99%

Efficient near-duplicate document detection using FPGAs

Luo

Najjar

Hristidis

2013

2013 IEEE International Conference on Big Data

View full text Add to dashboard Cite

Abstract-Detecting duplicate and near-duplicate documents is critical in applications like Web crawling since it helps save document processing resources. Simhash is a state-of-art method to assign a bit-string fingerprint to a document, such that similar documents have similar fingerprints. Finding the near-duplicates in a large collection of documents consists of two stages: (a) compute the simhash fingerprint of each document, and (b) find pairs of similar fingerprints by computing their Hamming distance.Previous work has focused on optimizing the second stage, i.e., avoiding the quadratic number of comparisons to compute the all to all Hamming distance. However, our experiments show that the total time is dominated by the first stage (the fingerprints computation), which is the focus of this paper. We propose an implementation of simhash on Field Programmable Gate Arrays (FPGAs), by implementing a customized fingerprint computing engine in hardware that exploits parallelization and pipelining opportunities. We present a comprehensive experimental evaluation on large diverse real document datasets. Our experiments show a speedup of 362x in the simhash computation, and savings of up to 98% in overall near-duplicate detection execution time compared to using multi-core CPUs.

show abstract

“…For a summary of dynamic hashing see Enbody and Du (1988) or Larson (1988). To briefly summarize dynamic hashing let a be the load factor, defined as the ratio of the number of elements in the matrix to the size of the hash table.…”

Section: Dynamic Hashingmentioning

confidence: 99%

Global Astrometric Solutions with Sparse Matrix Techniques

Branham¹

2000

International Astronomical Union Colloquium

View full text Add to dashboard Cite

Abstract. Modern astrometric techniques lead to large, linear systems solved by the precepts of least-squares. These systems are usually sparse, and one should take advantage of the sparsity to facilitate their solution. As long as the matrix A of the equations of condition possesses the weak Hall property, characteristic of linear systems derived from astrometric reductions, it is possible to find a sparse Cholesky factor. Before the equations of condition are accumulated, by use of the fast Givens transformation, a symbolic factorization of A using Tewarson's length of intersection technique determines the ordering of the columns of A that result in low fill-in. The non-null elements are stored in a sparse, dynamic data structure by use of dynamic hashing. Numerical experimentation shows that this competes well with alternatives such as nested dissection, and large, but sparse, linear systems with several thousand unknowns can be solved in a reasonable amount of time, even on personal computers.

show abstract

Dynamic hashing schemes

Cited by 101 publications

References 19 publications

Distributed Paged Hash Tables

Distributed Paged Hash Tables

Efficient near-duplicate document detection using FPGAs

Global Astrometric Solutions with Sparse Matrix Techniques

Contact Info

Product

Resources

About