Cloud Computing: A model Construct of Real-Time Monitoring for Big Dataset Analytics Using Apache Spark

Alkasem, Ameen; Liu, Hongwei; Zuo, Decheng; Algarash, Basheer

doi:10.1088/1742-6596/933/1/012018

Cited by 11 publications

(8 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Many of the research works have focused on developing machine learning models for the automatic scaling of network instances in the cloud. Time-series forecasting has been used in different research works to predict scaling decisions [18][19][20]. The cloud resource scaling for achieving energy efficiency is explained in [21].…”

Section: Related Workmentioning

confidence: 99%

An Intent-Driven Closed-Loop Platform for 5G Network Service Orchestration

Khan¹,

Abbas²,

Afaq³

et al. 2022

Computers, Materials &Amp; Continua

View full text Add to dashboard Cite

The scope of the 5G network is not only limited to the enhancements in the form of the quality of service (QoS), but it also includes a wide range of services with various requirements. Besides this, many approaches and platforms are under the umbrella of 5G to achieve the goals of endto-end service provisioning. However, the management of multiple services over heterogeneous platforms is a complex task. Each platform and service have various requirements to be handled by domain experts. Still, if the next-generation network management is dependent on manual updates, it will become impossible to provide seamless service provisioning in runtime. Since the traffic for a particular type of service varies significantly over time, automatic provisioning of resources and orchestration in runtime need to be integrated. Besides, with the increase in the number of devices, amount, and variety of traffic, the management of resources with optimization becomes a challenging task. To this end, this manuscript provides a solution that automates the management and service provisioning through multiple platforms while assuring various aspects, including automation, resource management and service assurance. The solution consists of an intent-based system that automatically manages different orchestrators, and eliminates manual control by abstracting the complex configuration requirements into simple and generic contracts. The proposed system considers handling the scalability of resources in runtime by using Machine Learning (ML) to automate and optimize service resource utilization.

show abstract

Section: Related Workmentioning

confidence: 99%

An Intent-Driven Closed-Loop Platform for 5G Network Service Orchestration

Khan¹,

Abbas²,

Afaq³

et al. 2022

Computers, Materials &Amp; Continua

View full text Add to dashboard Cite

show abstract

“…A study on VM failure prediction was carried out by Meenakumari et al [28], Alkasem et al [29], Qasem et al [30], Liu et al [31] and Rawat et al [32]. The study by Meenakumari et al [28] employed a dynamic thresholding approach to predict failure based on system metrics such as CPU utilisation, CPU usage, bandwidth, temperature, and memory.…”

Section: Vm-level Failure Predictionmentioning

confidence: 99%

“…The study by Meenakumari et al [28] employed a dynamic thresholding approach to predict failure based on system metrics such as CPU utilisation, CPU usage, bandwidth, temperature, and memory. Similar to Meenakumari et al [28], Alkasem et al [29] also focused on VM failure prediction. The study by Alkasem et al [29] focused on the VM startup failure problem by using system metrics such as CPU utilisation, memory usage, network overhead, and IO (input/output) storage usage.…”

Section: Vm-level Failure Predictionmentioning

confidence: 99%

“…Similar to Meenakumari et al [28], Alkasem et al [29] also focused on VM failure prediction. The study by Alkasem et al [29] focused on the VM startup failure problem by using system metrics such as CPU utilisation, memory usage, network overhead, and IO (input/output) storage usage. Alkasem et al [29] used Apache Spark [33] streaming together with Naïve Bayes (NB).…”

Section: Vm-level Failure Predictionmentioning

confidence: 99%

See 1 more Smart Citation

A Combined System Metrics Approach to Cloud Service Reliability Using Artificial Intelligence

Chhetri

Dehury

Lind

et al. 2022

BDCC

View full text Add to dashboard Cite

Identifying and anticipating potential failures in the cloud is an effective method for increasing cloud reliability and proactive failure management. Many studies have been conducted to predict potential failure, but none have combined SMART (self-monitoring, analysis, and reporting technology) hard drive metrics with other system metrics, such as central processing unit (CPU) utilisation. Therefore, we propose a combined system metrics approach for failure prediction based on artificial intelligence to improve reliability. We tested over 100 cloud servers’ data and four artificial intelligence algorithms: random forest, gradient boosting, long short-term memory, and gated recurrent unit, and also performed correlation analysis. Our correlation analysis sheds light on the relationships that exist between system metrics and failure, and the experimental results demonstrate the advantages of combining system metrics, outperforming the state-of-the-art.

show abstract

“…413 RTMBAS [27], FPHPC [19], DESH [22] and PSFML [21]. When we compare accuracy, 414 recall, precision and F1 score, we can see that all of our tested methods using combined 415 metrics outperform the existing state-of-the-art.…”

mentioning

confidence: 99%

A Combined Metrics Approach to Cloud Service Reliability using Artificial Intelligence

Chhetri¹,

Dehury²,

Lind³

et al. 2021

Preprint

View full text Add to dashboard Cite

Identifying and anticipating potential failures in the cloud is an effective method for increasing cloud reliability and proactive failure management. Many studies have been conducted to predict potential failure, but none have combined SMART (Self-Monitoring, Analysis, and Reporting Technology) hard drive metrics with other system metrics such as CPU utilisation. Therefore, we propose a combined metrics approach for failure prediction based on Artificial Intelligence to improve reliability. We tested over 100 cloud servers’ data and four AI algorithms: Random Forest, Gradient Boosting, Long-Short-Term Memory, and Gated Recurrent Unit. Our experimental result shows the benefits of combining metrics, outperforming state-of-the-art.

show abstract

Cloud Computing: A model Construct of Real-Time Monitoring for Big Dataset Analytics Using Apache Spark

Cited by 11 publications

References 13 publications

An Intent-Driven Closed-Loop Platform for 5G Network Service Orchestration

An Intent-Driven Closed-Loop Platform for 5G Network Service Orchestration

A Combined System Metrics Approach to Cloud Service Reliability Using Artificial Intelligence

A Combined Metrics Approach to Cloud Service Reliability using Artificial Intelligence

Contact Info

Product

Resources

About