A case study in building layered DHT applications

Abstract

Recent research has shown that one can use Distributed Hash Tables (DHTs) to build scalable, robust and efficient applications. One question that is often left unanswered is that of simplicity of implementation and deployment. In this paper, we explore a case study of building an application for which ease of deployment dominated the need for high performance. The application we focus on is Place Lab, an end-user positioning system. We evaluate whether it is feasible to use DHTs as an application-independent building block to implement a key component of Place Lab: its "mapping infrastructure." We present Prefix Hash Trees, a data structure used by Place Lab for geographic range queries that is built entire on top of a standard DHT. By strictly layering Place Lab's data structures on top of a generic DHT service, we were able to decouple the deployment and management of Place Lab from that of the underlying DHT. We identify the characteristics of Place Lab that made it amenable for deploying in this layered manner, and comment on its effect on performance.

GeoServ: A Distributed Urban Sensing Platform

“…However, since these systems use consistent hashing to map node ID and keyword to key space (i.e., DHT only provides exact match queries), it is non-trivial to support complex queries such as range queries. Chawathe et al [5] proposed a Prefix Hash Table (PHT) that is a trie-like data structure to provide a range query on top of the DHT layer. PIER [18], a distributed query engine based on DHTs provides rich declarative SQL queries such as equi-join.…”

Survey of DHT topology construction techniques in virtual computing environments

“…The PHT structure [37] supports several complex queries using a prefix hash tree, in which leaf nodes are keys and each internal node corresponds to a distinct prefix. It branches the leaves where attribute values are densely populated.…”

“…To address this problem, we have proposed an efficient distributed index construction technique, the balanced Kautz tree (BK tree) [55], based on the DK DHT. The BK tree realizes mapping from resource space onto node space using the Z-curve [56], and creates an efficient resource information indexing structure based on the PHT technique [37]. Each inner node of the BK tree has d sub-nodes, where d is called base of the BK tree.…”

Scalable P2P Reconciliation Infrastructure for Collaborative Text Editing

“…(correct lookup service): Like several other protocols and applications designed over DHTs, e.g. [4], in our work we assume that the lookup service of the DHT behaves properly. That is, given a key k it either finds correctly the responsible for k or reports an error, e.g.…”