Tuning Paxos for High-Throughput with Batching and Pipelining

Santos, Norberto; Schiper, André

doi:10.1007/978-3-642-25959-3_11

Cited by 25 publications

(19 citation statements)

References 13 publications

(26 reference statements)

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Two optimizations that greatly improve the performance of Paxos are batching and pipelining [12]. Batching consists of grouping several client requests in the same ballot, while pipelining consists of executing several ballots concurrently.…”

Section: A State Machine Replicationmentioning

confidence: 99%

Achieving High-Throughput State Machine Replication in Multi-core Systems

Santos

Schiper

2013

2013 IEEE 33rd International Conference on Distributed Computing Systems

Self Cite

View full text Add to dashboard Cite

Abstract-State machine replication is becoming an increasingly popular technique among online services to ensure faulttolerance using commodity hardware. This has led to a renewed interest in its throughput, as these services have typically a large number of users. Recent work has shown how to improve throughput of the ordering phase by improving the replication protocol, using techniques like Ring-topologies, IP multicast, and rotating leaders. The resulting systems, when deployed in modern fast networks, achieve unprecedented levels of throughput. But these systems are increasingly becoming limited by the CPU of the replicas, especially with small client requests. However, the problem is not lack of performance of the CPUs, but instead the inability of typical implementations to effectively use the multiple cores of modern multi-core CPUs. In this work, we show how to architect a replicated state machine whose performance scales with the number of cores. We do so by applying several good practices of concurrent programming to the specific case of state machine replication, including staged execution, workload partitioning, actors, and non-blocking data structures. We describe and test a Java prototype of our architecture, based on the Paxos protocol. With a workload consisting of small requests, we achieve a 6 times improvement in throughput using 8 cores. More generally, in all our experiments we have consistently reached the limits of network subsystem by using up to 12 cores, and do not observe any degradation when using up to 24 cores. Furthermore, the profiling results of our implementation show that even at peak throughput contention between threads is minimal, suggesting that the throughput would continue scaling given a faster network.

show abstract

Section: A State Machine Replicationmentioning

confidence: 99%

Achieving High-Throughput State Machine Replication in Multi-core Systems

Santos

Schiper

2013

2013 IEEE 33rd International Conference on Distributed Computing Systems

Self Cite

View full text Add to dashboard Cite

show abstract

“…More precisely, the network subsystem at the leader can reach its maximum capacity in terms of packets per second [19], [20]. In this case, we say the system is network subsystem bound.…”

Section: Limitations Of Paxosmentioning

confidence: 99%

S-Paxos: Offloading the Leader for High Throughput State Machine Replication

Biely

Milošević

Santos

et al. 2012

2012 IEEE 31st Symposium on Reliable Distributed Systems

Self Cite

View full text Add to dashboard Cite

Abstract-Implementations of state machine replication are prevalently using variants of Paxos or other leader-based protocols. Typically these protocols are also leader-centric, in the sense that the leader performs more work than the non-leader replicas. Such protocols scale poorly, because as the number of replicas or the load on the system increases, the leader replica quickly reaches the limits of one of its resources. In this paper we show that much of the work performed by the leader in a leadercentric protocol can in fact be evenly distributed among all the replicas, thereby leaving the leader only with minimal additional workload. This is done (i) by distributing the work of handling client communication among all replicas, (ii) by disseminating client requests among replicas in a distributed fashion, and (iii) by executing the ordering protocol on ids. We derive a variant of Paxos incorporating these ideas. Compared to leader-centric protocols, our protocol not only achieves significantly higher throughput for any given number of replicas, but also increases its throughput with the number of replicas.

show abstract

“…Besides providing insight into the difficulty of implementing Paxos, two of these papers present performance results of their Paxos implementations. In [15], an analytical analysis of the impact of several optimizations on the performance of the basic Paxos algorithm is presented. The authors also present extensive experimental results of their implementation in both LAN and WAN environments.…”

Section: Related Workmentioning

confidence: 99%