Addressing the ZooKeeper Synchronization Inefficiency

Abstract

Abstract. In this paper we discuss the problem of synchronization in ZooKeeper, a fault-tolerant distributed coordination framework. One of the key features of ZooKeeper is to move away from blocking API such as locks, in order to avoid problems with slow or faulty clients. Instead, it provides an event like synchronization mechanism, allowing clients to be notified upon state change on the server. However, such a mechanism leads to very inefficient implementation of synchronization objects such as queues or barriers. We propose a new solution to this problem. The solution is to handle a sequence of client operations completely on the server. This means that the client implements the required sequence of operations as a single request, which is sent to the server for execution via a generic API. We present a prototype that shares some of the concepts of ZooKeeper but, contrary to ZooKeeper, allows a very efficient implementation of synchronization objects. The solution requires a deterministic multi-threaded server, which we implement thanks to a coroutine mechanism. Experiments show the significant gain in efficiency of our solution on producer-consumer queues and synchronization barriers.

A framework for microservices synchronization

“…The Zookeeper's synchronization mechanism is limited as the update notification does not carry the updated values and it suffers of some inefficiencies. 32 Consul, 33 based on an implementation of the Serf 34 algorithm, combines service discovery features along with distributed configuration management. It embeds a strongly consistent key-value store, robust monitoring, and health checking.…”

“…It exposes common services for naming, configuration management, and synchronization. The Zookeeper's synchronization mechanism is limited as the update notification does not carry the updated values and it suffers of some inefficiencies 32 . Consul, 33 based on an implementation of the Serf 34 algorithm, combines service discovery features along with distributed configuration management.…”

A Stochastic Adjustment Strategy for Coordination Process in Distributed Networks

“…Researchers have expended considerable effort to improve the ZooKeeper's performance. Kalantari and Schiper [11] developed a prototype to reduce the synchronization time between ZooKeeper servers. Junqueira et al [12] designed a crashrecovery atomic broadcast algorithm to guarantee state coordination.…”

Extensible distributed coordination

“…Several works have presented modifications and additions to ZooKeeper (e.g., [21,23,25,36,40,55]), but (almost) none of them deals with changing the service's programming model. A notable exception is a recent short paper by Kalantari et al [36] which identifies inefficiencies related to ZooKeeper's watch mechanism. As a solution, the authors present the prototype implementation of a coordination service that is able to execute critical sections (i.e., sequences of operations that are protected by a lock) relying on a deterministic multi-threaded server.…”