Asymmetric-partition replication for highly scalable distributed transaction processing in practice

Lee, Juchang; Kim, Kyu Hwan; Lee, Hyejeong; Andrei, Mihnea; Ko, Seongyun; Keller, F.; Han, Wook-Shin

doi:10.14778/3415478.3415538

Cited by 6 publications

(3 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…On the other hand, asymmetric partitioning has the potential to generate a more efficient partitioning strategy by customizing the strategy for each data store and workload independently. For instance, SAP HANA [96] maintains a row store in a single physical machine without partitioning, while its column store is independently partitioned and distributed across multiple smaller physical machines. This enables it to avoid the cross-partition two-phase commit [119] (2PC) for OLTP workloads while serving partitionfriendly OLAP workloads in a more scalable way.…”

Section: Sharding Strategymentioning

confidence: 99%

A survey on hybrid transactional and analytical processing

Song,

Zhou,

Cui

et al. 2024

The VLDB Journal

View full text Add to dashboard Cite

To provide applications with the ability to analyze fresh data and eliminate the time-consuming ETL workflow, hybrid transactional and analytical (HTAP) systems have been developed to serve online transaction processing and online analytical processing workloads in a single system. In recent years, HTAP systems have attracted considerable interest from both academia and industry. Several new architectures and technologies have been proposed. This paper provides a comprehensive overview of these HTAP systems. We review recently published papers and technical reports in this field and broadly classify existing HTAP systems into two categories based on their data formats: monolithic and hybrid HTAP. We further classify hybrid HTAP into four sub-categories based on their storage architecture: row-oriented, column-oriented, separated, and hybrid. Based on such a taxonomy, we outline each stream’s design challenges and performance issues (e.g., the contradictory format demand for monolithic HTAP). We then discuss potential solutions and their trade-offs by reviewing noteworthy research findings. Finally, we summarize emerging HTAP applications, benchmarks, future trends, and open problems.

show abstract

Section: Sharding Strategymentioning

confidence: 99%

A survey on hybrid transactional and analytical processing

Song,

Zhou,

Cui

et al. 2024

The VLDB Journal

View full text Add to dashboard Cite

show abstract

“…Object partitioning has been used to improve performance of distributed transactions. Typically, objects are partitioned and migrated periodically to improve locality [1,17,23,37,39,54,61,66]. In geo-distributed systems, object migration can significantly reduce WAN traffic [14].…”

Section: Related Workmentioning

confidence: 99%

Zeus

Katsarakis

Tan

et al. 2021

Proceedings of the Sixteenth European Conference on Computer Systems

View full text Add to dashboard Cite

State-of-the-art distributed in-memory datastores (FaRM, FaSST, DrTM) provide strongly-consistent distributed transactions with high performance and availability. Transactions in those systems are fully general; they can atomically manipulate any set of objects in the store, regardless of their location. To achieve this, these systems use complex distributed transactional protocols. Meanwhile, many workloads have a high degree of locality. For such workloads, distributed transactions are an overkill as most operations only access objects located on the same server -if sharded appropriately.In this paper, we show that for these workloads, a singlenode transactional protocol combined with dynamic object re-sharding and asynchronously pipelined replication can provide the same level of generality with better performance, simpler protocols, and lower developer effort. We present Zeus, an in-memory distributed datastore that provides general transactions by acquiring all objects involved in the transaction to the same server and executing a single-node transaction on them. Zeus is fault-tolerant and stronglyconsistent. At the heart of Zeus is a reliable dynamic object sharding protocol that can move 250K objects per second per server, allowing Zeus to process millions of transactions per second and outperform more traditional distributed transactions on a wide range of workloads that exhibit locality.

show abstract

“…The existing methodologies in data partitioning are Hash, Round-robin, List and Range partitioning [17]. These approaches are not capable for the increasing number of transaction loads.…”

Section: Introductionmentioning

confidence: 99%

A Novel HVgP and OLBP approach in Distributed Transaction Partitioning

Bharati

Attar

2022

Preprint

View full text Add to dashboard Cite

Data partitioning acts an important role in Online Transaction Processing (OLTP) to enhance their performance and scalability. Partitioning is important in OLTP based systems due to the increase of user populations and online transactions. Nowadays, providing scalable and reliable data organization is the main aim of database researchers. This work presented an innovative data partitioning approach and optimized load balancing for scalable transactions. Initially, the Hybrid Vertical-graph partitioning (HVgP) approach is utilized for appropriate data partitioning. After the process of data partitioning, the partitions efficiency and the average workload parameters are computed and their corresponding weight factor is computed. Subsequently, Optimized Improved Load Balancing (OLBP) algorithm is utilized for scalable OLTP data transactions on distributed database. Here, the estimated weight factor is updated for effective load balancing with data partitions. The presented approach is appropriate for various online data transaction applications. The quality of the presented approach is examined using TPC-E OLTP benchmark dataset. The performance of the improved approach is examined with the different existing approaches and proved the significant enhancement in different performance metrics like Throughput, Response time, distributed transactions, and CPU utilization.

show abstract

Asymmetric-partition replication for highly scalable distributed transaction processing in practice

Cited by 6 publications

References 14 publications

A survey on hybrid transactional and analytical processing

A survey on hybrid transactional and analytical processing

Zeus

A Novel HVgP and OLBP approach in Distributed Transaction Partitioning

Contact Info

Product

Resources

About