Migration-Based Online CPSCN Big Data Analysis in Data Centers

Li, Xin; Wang, Liangyuan; Lian, Zhen; Qin, Xiaolin

doi:10.1109/access.2018.2810255

Cited by 7 publications

(15 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Additionally, in the task response time, we consider the replicated datasets, the computational capacity, and the load of each server to prevent underloaded and overloaded machines. Finally, after comparing our online task scheduling OTS-DMDR with other existing algorithms in the literature, the corresponding results show that the proposed OTS-DMDR can guarantee better average response time by 46% compared to Li et al [28], by 58% compared to the Delay Scheduling method, and acceptable load balancing between machines, improving the overall system efficiency.…”

Section: Introductionmentioning

confidence: 95%

“…However, as scientific applications become more and more data-intensive, handling storage, data management, and computing resources is increasingly critical [65]. The most related scheduling strategy to our work is presented by Li et al in [28]. They proposed an online job scheduling based on data migration by selecting a proper task to be scheduled when a server becomes available.…”

Section: Our Motivationmentioning

confidence: 99%

“…Paper [28] schedules tasks sequentially, one task after the other, which increases the waiting time for tasks in the queue. Moreover, when migrating data, a set of characteristics were not considered such as machine performance, the network between machines, storage space, and task requirements in terms of CPU, RAM, size, and volume of required data per task.…”

Section: Our Motivationmentioning

confidence: 99%

See 2 more Smart Citations

Online Task Scheduling of Big Data Applications in the Cloud Environment

2023

View full text Add to dashboard Cite

The development of big data has generated data-intensive tasks that are usually time-consuming, with a high demand on cloud data centers for hosting big data applications. It becomes necessary to consider both data and task management to find the optimal resource allocation scheme, which is a challenging research issue. In this paper, we address the problem of online task scheduling combined with data migration and replication in order to reduce the overall response time as well as ensure that the available resources are efficiently used. We introduce a new scheduling technique, named Online Task Scheduling algorithm based on Data Migration and Data Replication (OTS-DMDR). The main objective is to efficiently assign online incoming tasks to the available servers while considering the access time of the required datasets and their replicas, the execution time of the task in different machines, and the computational power of each machine. The core idea is to achieve better data locality by performing an effective data migration while handling replicas. As a result, the overall response time of the online tasks is reduced, and the throughput is improved with enhanced machine resource utilization. To validate the performance of the proposed scheduling method, we run in-depth simulations with various scenarios and the results show that our proposed strategy performs better than the other existing approaches. In fact, it reduces the response time by 78% when compared to the First Come First Served scheduler (FCFS), by 58% compared to the Delay Scheduling, and by 46% compared to the technique of Li et al. Consequently, the present OTS-DMDR method is very effective and convenient for the problem of online task scheduling.

show abstract

Section: Introductionmentioning

confidence: 95%

Section: Our Motivationmentioning

confidence: 99%

Section: Our Motivationmentioning

confidence: 99%

See 1 more Smart Citation

Online Task Scheduling of Big Data Applications in the Cloud Environment

2023

View full text Add to dashboard Cite

show abstract

“…The joint task and data placement is another approach to achieve data locality for offline tasks [24], which analyzes the relevancy between task and data to decide how to arrange the data placement and task assignment comprehensively. In addition, with the different idea, the migration approach [7] is proposed to achieve data locality actively. The main idea is to compare the cost between waiting cost and migration cost.…”

Section: Related Workmentioning

confidence: 99%

“…However, some traditional task scheduling methods are no longer applicable to heterogeneous tasks like FIFO. To satisfy the demands of time reduction, in our algorithm, we first established a time model for time calculation of tasks executed in different modes based on some of our previous work [7]. In order to execute tasks in locality mode as much as possible, we migrate the required data block to the server when scheduling a task.…”

Section: Introductionmentioning

confidence: 99%

Data-Intensive Task Scheduling for Heterogeneous Big Data Analytics in IoT System

Wang

Abawajy

et al. 2020

Energies

Self Cite

View full text Add to dashboard Cite

Efficient big data analysis is critical to support applications or services in Internet of Things (IoT) system, especially for the time-intensive services. Hence, the data center may host heterogeneous big data analysis tasks for multiple IoT systems. It is a challenging problem since the data centers usually need to schedule a large number of periodic or online tasks in a short time. In this paper, we investigate the heterogeneous task scheduling problem to reduce the global task execution time, which is also an efficient method to reduce energy consumption for data centers. We establish the task execution for heterogeneous tasks respectively based on the data locality feature, which also indicate the relationship among the tasks, data blocks and servers. We propose a heterogeneous task scheduling algorithm with data migration. The core idea of the algorithm is to maximize the efficiency by comparing the cost between remote task execution and data migration, which could improve the data locality and reduce task execution time. We conduct extensive simulations and the experimental results show that our algorithm has better performance than the traditional methods, and data migration actually works to reduce th overall task execution time. The algorithm also shows acceptable fairness for the heterogeneous tasks.

show abstract

Data-Centric Task Scheduling Algorithm for Hybrid Tasks in Cloud Data Centers

Wang

Abawajy

et al. 2018

Algorithms and Architectures for Parallel Processing

View full text Add to dashboard Cite

Migration-Based Online CPSCN Big Data Analysis in Data Centers

Cited by 7 publications

References 30 publications

Online Task Scheduling of Big Data Applications in the Cloud Environment

Online Task Scheduling of Big Data Applications in the Cloud Environment

Data-Intensive Task Scheduling for Heterogeneous Big Data Analytics in IoT System

Data-Centric Task Scheduling Algorithm for Hybrid Tasks in Cloud Data Centers

Contact Info

Product

Resources

About