RENDA: Resource and Network Aware Data Placement Algorithm for Periodic Workloads in Cloud

Thakkar, Hiren Kumar; Sahoo, Prasan Kumar; Veeravalli, Bharadwaj

doi:10.1109/tpds.2021.3080582

Cited by 22 publications

(13 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It also has limited scalability as it is table based approach, and heavy computation analysis on user's interaction and data access pattern is required. RENDA [10] uses computation resources, network latency, and bandwidth to optimize data placement for periodic workloads in the cloud storage system. However, as it is optimized for recurrent workloads, it is unsuitable for generic data distribution methods.…”

Section: Literature Reviewmentioning

confidence: 99%

“…Methods for distributing data in a distributed storage system include table-based [4][5][6][7][8][9][10], hashbased [11][12][13][14][15][16][17][18], and subtree-based [19][20][21][22][23] methods. Table-based data distribution technology manages one central table in multiple servers, and the central table maintains data information and identifiers for data storage.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Data Utilization-Based Adaptive Data Management Method for Distributed Storage System in WAN Environment

Lee¹,

Kim²,

Jo³

et al. 2023

Computer Systems Science and Engineering

View full text Add to dashboard Cite

Recently, research on a distributed storage system that efficiently manages a large amount of data has been actively conducted following data production and demand increase. Physical expansion limits exist for traditional standalone storage systems, such as I/O and file system capacity. However, the existing distributed storage system does not consider where data is consumed and is more focused on data dissemination and optimizing the lookup cost of data location. And this leads to system performance degradation due to low locality occurring in a Wide Area Network (WAN) environment with high network latency. This problem hinders deploying distributed storage systems to multiple data centers over WAN. It lowers the scalability of distributed storage systems to accommodate data storage needs. This paper proposes a method for distributing data in a WAN environment considering network latency and data locality to solve this problem and increase overall system performance. The proposed distributed storage method monitors data utilization and locality to classify data temperature as hot, warm, and cold. With assigned data temperature, the proposed algorithm adaptively selects the appropriate data center and places data accordingly to overcome the excess latency from the WAN environment, leading to overall system performance degradation. This paper also conducts simulations to evaluate the proposed and existing distributed storage methods. The result shows that our proposed method reduced latency by 38% compared to the existing method. Therefore, the proposed method in this paper can be used in large-scale distributed storage systems over a WAN environment to improve latency and performance compared to existing methods, such as consistent hashing.

show abstract

Section: Literature Reviewmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Data Utilization-Based Adaptive Data Management Method for Distributed Storage System in WAN Environment

Lee¹,

Kim²,

Jo³

et al. 2023

Computer Systems Science and Engineering

View full text Add to dashboard Cite

show abstract

“…In recent years, there have been countless applications of machine learning [ 19 ] and reinforcement learning [ 20 ] in the diversified areas such as healthcare predictions [ 21 ], cloud resource management [ 22 ], and mobile robot navigation [ 23 ]. Moreover, a significant surge is also observed in cyber frauds, as well as the corresponding model to counter them, such as credit card fraud detection, telecom churn prediction [ 2 – 5 ], and detecting rare medical diseases.…”

Section: Related Workmentioning

confidence: 99%

Clairvoyant: AdaBoost with Cost-Enabled Cost-Sensitive Classifier for Customer Churn Prediction

Thakkar

Desai

Ghosh³

et al. 2022

Computational Intelligence and Neuroscience

Self Cite

View full text Add to dashboard Cite

Customer churn prediction is one of the challenging problems and paramount concerns for telecommunication industries. With the increasing number of mobile operators, users can switch from one mobile operator to another if they are unsatisfied with the service. Marketing literature states that it costs 5–10 times more to acquire a new customer than retain an existing one. Hence, effective customer churn management has become a crucial demand for mobile communication operators. Researchers have proposed several classifiers and boosting methods to control customer churn rate, including deep learning (DL) algorithms. However, conventional classification algorithms follow an error-based framework that focuses on improving the classifier’s accuracy over cost sensitization. Typical classification algorithms treat misclassification errors equally, which is not applicable in practice. On the contrary, DL algorithms are computationally expensive as well as time-consuming. In this paper, a novel class-dependent cost-sensitive boosting algorithm called AdaBoostWithCost is proposed to reduce the churn cost. This study demonstrates the empirical evaluation of the proposed AdaBoostWithCost algorithm, which consistently outperforms the discrete AdaBoost algorithm concerning telecom churn prediction. The key focus of the AdaBoostWithCost classifier is to reduce false-negative error and the misclassification cost more significantly than the AdaBoost.

show abstract

“…Their approach is based on the node's computing capabilities to improve the map tasks' data locality and to reduce the job turnaround time in the heterogeneous Hadoop environment. Resource-and Network-aware Data Placement Algorithm (RENDA) for resource-and network-aware data placement in Hadoop is presented in [12]. The RENDA reduces the time of the data distribution and data processing stages by estimating the heterogeneous performance of the nodes on a real-time basis.…”

Section: Introductionmentioning

confidence: 99%

Adaptive Multi-Criteria Selection for Efficient Resource Allocation in Frugal Heterogeneous Hadoop Clusters

Qureshi

2024

Electronics

View full text Add to dashboard Cite

Efficient resource allocation is crucial in clusters with frugal Single-Board Computers (SBCs) possessing limited computational resources. These clusters are increasingly being deployed in edge computing environments in resource-constrained settings where energy efficiency and cost-effectiveness are paramount. A major challenge in Hadoop scheduling is load balancing, as frugal nodes within the cluster can become overwhelmed, resulting in degraded performance and frequent occurrences of out-of-memory errors, ultimately leading to job failures. In this study, we introduce an Adaptive Multi-criteria Selection for Efficient Resource Allocation (AMS-ERA) in Frugal Heterogeneous Hadoop Clusters. Our criterion considers CPU, memory, and disk requirements for jobs and aligns the requirements with available resources in the cluster for optimal resource allocation. To validate our approach, we deploy a heterogeneous SBC-based cluster consisting of 11 SBC nodes and conduct several experiments to evaluate the performance using Hadoop wordcount and terasort benchmark for various workload settings. The results are compared to the Hadoop-Fair, FOG, and IDaPS scheduling strategies. Our results demonstrate a significant improvement in performance with the proposed AMS-ERA, reducing execution time by 27.2%, 17.4%, and 7.6%, respectively, using terasort and wordcount benchmarks.

show abstract

RENDA: Resource and Network Aware Data Placement Algorithm for Periodic Workloads in Cloud

Cited by 22 publications

References 30 publications

Data Utilization-Based Adaptive Data Management Method for Distributed Storage System in WAN Environment

Data Utilization-Based Adaptive Data Management Method for Distributed Storage System in WAN Environment

Clairvoyant: AdaBoost with Cost-Enabled Cost-Sensitive Classifier for Customer Churn Prediction

Adaptive Multi-Criteria Selection for Efficient Resource Allocation in Frugal Heterogeneous Hadoop Clusters

Contact Info

Product

Resources

About