A Read-Only Distributed Hash Table

Abstract

Distributed hash table (DHT) is an infrastructure to support resource discovery in large distributed system. In DHT, data items are distributed across an overlay network based on a hash function. This leads to two major issues. Firstly, to preserve ownership of data items, commercial applications may not allow a node to proactively store its data items on other nodes. Secondly, data-item distribution requires all nodes in a DHT overlay to be publicly writable, but some nodes do not permit the sharing of its storage to external parties due to a different economical interest. In this thesis, we present a DHT-based resource discovery scheme without distributing data items called R-DHT (Read-only DHT). We further extend R-DHT to support multi-attribute queries with our Midas scheme (Multi-dimensional range queries).R-DHT is a new DHT abstraction that does not distribute data items across an overlay network. To map each data item (e.g. a resource, an index to a resource, or resource metadata) back onto its resource owner (i.e. physical host), we virtualize each host into virtual nodes. These nodes are further organized as a segment-based overlay network with each segment consisting of resources of the same type. The segment-based overlay also increases R-DHT resiliency to node failures. Compared to conventional DHT, R-DHT's overlay has a higher number of nodes which increases lookup path length and maintenance overhead. To reduce iii R-DHT lookup path length, we propose various optimizations, namely routing by segments and shared finger tables. To reduce the maintenance overhead of overlay networks, we propose a hierarchical R-DHT which organizes nodes as a two-level overlay network. The top-level overlay is indexed based on resource types and constitutes the entry points for resource owners at second-level overlays.Midas is a scheme to support multi-attribute queries on R-DHT based on d-toone mapping. A multi-attribute resource is indexed by a one-dimensional key which is derived by applying a Hilbert space-filling curve (SFC) to the type of the resource. The resource is then mapped (i.e. virtualized) onto an R-DHT node. To retrive query results, a multi-attribute query is transformed into a number of exact queries using Hilbert SFC. These exact queries are further processed using R-DHT lookups. To reduce the number of lookup required, we propose two optimizations to Midas query engine, namely incremental search and search-key elimination.We evaluate R-DHT and Midas through analytical and simulation analysis. Our main findings are as follows. Firstly, the lookup path length of each R-DHT lookup operation is indeed independent of the number of virtual nodes. This demonstrates that our lookup optimization techniques are applicable to other DHT-based systems that also virtualize physical hosts into nodes. Secondly, we found that R-DHT is effective in supporting multi-attribute range queries when the number of query results is small. Our results also imply that a selective data-item distribution scheme would red...