ZHT: A Light-Weight Reliable Persistent Dynamic Scalable Zero-Hop Distributed Hash Table

Abstract

Abstract-This paper presents ZHT, a zero-hop distributed hash table, which has been tuned for the requirements of high-end computing systems. ZHT aims to be a building block for future distributed systems, such as parallel and distributed file systems, distributed job management systems, and parallel programming systems. The goals of ZHT are delivering high availability, good fault tolerance, high throughput, and low latencies, at extreme scales of millions of nodes. ZHT has some important properties, such as being light-weight, dynamically allowing nodes join and leave, fault tolerant through replication, persistent, scalable, and supporting unconventional operations such as append (providing lock-free concurrent key/value modifications) in addition to insert/lookup/remove. We have evaluated ZHT's performance under a variety of systems, ranging from a Linux cluster with 512-cores, to an IBM Blue Gene/P supercomputer with 160K-cores. Using micro-benchmarks, we scaled ZHT up to 32K-cores with latencies of only 1.1ms and 18M operations/sec throughput. This work provides three real systems that have integrated with ZHT, and evaluate them at modest scales. 1) ZHT was used in the FusionFS distributed file system to deliver distributed meta-data management at over 60K operations (e.g. file create) per second at 2K-core scales. 2) ZHT was used in the IStore, an information dispersal algorithm enabled distributed object storage system, to manage chunk locations, delivering more than 500 chunks/sec at 32-nodes scales. 3) ZHT was also used as a building block to MATRIX, a distributed job scheduling system, delivering 5000 jobs/sec throughputs at 2K-core scales. We compared ZHT against other distributed hash tables and key/value stores and found it offers superior performance for the features and portability it supports.

Overcoming data locality: An in-memory runtime file system with symmetrical data distribution

“…This system can be used in conjunction with SPADE [21] to capture data provenance with negligible overheads [22]. Distributed metadata storage is implemented by means of ZHT [23] which leverages excellent scalability.…”

Coverage Area Optimization Algorithm for Directed Sensing Network Based on Energy Aware

“…These nodes are statically deployed in the target area, namely: such WSN is a static network. Additionally, the sensor nodes can obtain their own position information through positioning technology [12].…”

Achieving Efficient Distributed Scheduling with Message Queues in the Cloud for Many-Task Computing and High-Performance Computing

“…We have been prototyping distributed job launch in the Slurm job resource manager under a system called Slurm++ [32], but that work is not mature enough yet to be included in this study. Moreover, CloudKon is the only distributed task scheduler that is designed and optimized to run on public cloud environment.…”

“…We are going to implement a SQS like service with high throughput at the larger access scales. With help from other systems such as ZHT distributed hash table [32], we will be able implement such a service. Another future direction of this work is to implement a more tightly coupled version of CloudKon and test it on supercomputers and HPC environments while running HPC jobs in a distributed fashion, and to compare it directly with Slurm and Slurm++ in the same environment.…”