Load balancing scheme for supporting real-time processing of big data in distributed in-memory systems

Bok, Kyoungsoo; Choi, Kitae; Lim, Jongtae; Yoo, Jaesoo

doi:10.1145/3264746.3264768

Cited by 1 publication

(5 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To address such a load imbalance among nodes in the distributed in-memory environments, studies have been conducted using ring-based hashing schemes [32][33][34][35][36][37]. The general ring-based hashing schemes adjust the loading by replicating the data to other nodes, or by migrating the data through a hash space adjustment.…”

Section: Introductionmentioning

confidence: 99%

“…We propose a load balancing scheme in distributed in-memory [37]. The authors of [37] proposed the concept of load balancing using data replication and data migration in a distributed in-memory environment. However, Ref.…”

Section: Introductionmentioning

confidence: 99%

“…However, Ref. [37] did not provide initial data distribution techniques in heterogeneous distributed in-memory environments and did not check the load status of an overloaded node after data replication and data migration. In addition, load balancing due to the addition and removal of nodes was not performed for distributed in-memory processing.…”

Section: Introductionmentioning

confidence: 99%

“…In addition, load balancing due to the addition and removal of nodes was not performed for distributed in-memory processing. In this paper, we propose a new load balancing technique that extends the existing scheme [37] in heterogeneous distributed in-memory environments. The proposed scheme distinguishes the load status according to whether hot data is generated and performs data replication or data migration using a load balancer.…”

Section: Introductionmentioning

confidence: 99%

“…The clients maintain up-to-date metadata by periodically synchronizing the metadata information about accessible data with the load balancer. The main contributions of this paper over [37] are as follows:…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Load Balancing Scheme for Effectively Supporting Distributed In-Memory Based Computing

Bok

Choi

et al. 2019

Electronics

Self Cite

View full text Add to dashboard Cite

As digital data have increased exponentially due to an increasing number of information channels that create and distribute the data, distributed in-memory systems were introduced to process big data in real-time. However, when the load is concentrated on a specific node in a distributed in-memory environment, the data access performance is degraded, resulting in an overall degradation in the processing performance. In this paper, we propose a new load balancing scheme that performs data migration or replication according to the loading status in heterogeneous distributed in-memory environments. The proposed scheme replicates hot data when the hot data occurs on the node where a load occurs. If the load of the node increases in the absence of hot data, the data is migrated through a hash space adjustment. In addition, when nodes are added or removed, data distribution is performed by adjusting the hash space with the adjacent nodes. The clients store the metadata of the hot data and reduce the access of the load balancer through periodic synchronization. It is confirmed through various performance evaluations that the proposed load balancing scheme improves the overall load balancing performance.

show abstract

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations