A New Approach for Scheduling Tasks and/or Jobs in Big Data Cluster

Hadjar, Karim; Jedidi, Ahmed

doi:10.1109/icbdsc.2019.8645613

Cited by 11 publications

(9 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…(5) Our strategy only investigates on the scheduling of computing tasks. However, a GPU‐enabled cluster can be the underlying infrastructure of many kinds of distributed systems including HBase, Spark, and so on 33 . Our future work will take into consideration resource loads of I/O and memory.…”

Section: Discussionmentioning

confidence: 99%

Adaptive and transparent task scheduling of GPU‐powered clusters

Yang

et al. 2020

Concurrency and Computation

View full text Add to dashboard Cite

GPGPU-powered supercomputers are vital for various science and engineering applications. On each cluster node, the GPU works as a coprocessor of the CPU, and the computing task runs alternatively on CPU and GPU. Due to this characteristic, traditional task scheduling strategy tends to result in significant workload imbalance and underutilization of GPUs. We design an adaptive scheduling strategy to alleviate such imbalance and underutilization. Our strategy proposes to logically treats all GPUs in the cluster as a whole. Every cluster node maintains a local information table of all GPUs. Once a GPU call request is received, a node selects a GPU to run the task in an adaptive manner based on this table. In addition, our strategy does not rely on a global queue, and thus avoids excessive internode communication overhead. Moreover, we encapsulate our strategy into an intermedia module between the cluster and users. Consequently, underlying details of task scheduling is transparent to users, which enhances usability. We validate our strategy through experiments.

show abstract

Section: Discussionmentioning

confidence: 99%

Adaptive and transparent task scheduling of GPU‐powered clusters

Yang

et al. 2020

Concurrency and Computation

View full text Add to dashboard Cite

show abstract

“…Author in [7] suggested a new technique to schedule the Jobs or tasks in Big Data cluster. The uniqueness of this proposed method is that it basically focuses on the resources utilization and the type of Scheduled job altogether.…”

Section: Literature Review On Task Scheduling In Fog-cloud Environmentmentioning

confidence: 99%

An Adaptive Task Scheduling in Fog Computing

Harkut¹,

Thakar²,

Mutneja³

2023

Search Algorithm - Essence of Optimization

View full text Add to dashboard Cite

Internet applications generate massive amount of data. For processing the data, it is transmitted to cloud. Time-sensitive applications require faster access. However, the limitation with the cloud is the connectivity with the end devices. Fog was developed by Cisco to overcome this limitation. Fog has better connectivity with the end devices, with some limitations. Fog works as intermediate layer between the end devices and the cloud. When providing the quality of service to end users, scheduling plays an important role. Scheduling a task based on the end users requirement is a tedious thing. In this paper, we proposed a cloud-fog task scheduling model, which provides quality of service to end devices with proper security.

show abstract

“…However, researchers must proceed cautiously while adopting the DTSS scheduler because it is ineffective when jobs have convenient access to datalocal nodes. This paper [56] suggested a new method for scheduling tasks and/or jobs in a Big Data Cluster that focuses on enhancing the NameNode's task distribution to the data nodes. This task scheduler outperforms the existing task schedulers: FIFO Scheduler and Capacity Scheduler, as evidenced by the significant results gained.…”

Section: Other Improved Algorithmsmentioning

confidence: 99%

“…They define the work packing problem, which incorporates maximizing the job sets' completion efficiency, and deploy the k-means clustering technique to achieve JCT alignment. When the k-means clustering approach yields excessive groups, the number of jobs in each job unit decreases, which is a disadvantage for the job packing method to pick and pack appropriate jobs from a job group [56]. By integrating Round robin and the priority scheduling technique, a hybrid scheduler is presented in [58].…”

Section: Other Improved Algorithmsmentioning

confidence: 99%

An Overview of Hadoop Job Scheduling Algorithms for Big Data

Zameel

Zengi̇n

2022

Mugla Journal of Science and Technology

View full text Add to dashboard Cite

Rapid advancements in Big data systems have occurred over the last several decades. The significant element for attaining high performance is "Job Scheduling" in Big data systems which requires more utmost attention to resolve some challenges of scheduling. To obtain higher performance when processing the big data, proper scheduling is required. Apache Hadoop is most commonly used to manage immense data volumes in an efficient way and also proficient in handling the issues associated with job scheduling. To improve performance of big data systems, we significantly analyzed various Hadoop job scheduling algorithms. To get an overall idea about the scheduling algorithm, this paper presents a rigorous background. This paper made an overview on the fundamental architecture of Hadoop Big data framework, job scheduling and its issues, then reviewed and compared the most important and fundamental Hadoop job scheduling algorithms. In addition, this paper includes a review of other improved algorithms. The primary objective is to present an overview of various scheduling algorithms to improve performance when analyzing big data. This study will also provide appropriate direction in terms of job scheduling algorithm to the researcher according to which characteristics are most significant

show abstract

A New Approach for Scheduling Tasks and/or Jobs in Big Data Cluster

Cited by 11 publications

References 9 publications

Adaptive and transparent task scheduling of GPU‐powered clusters

Adaptive and transparent task scheduling of GPU‐powered clusters

An Adaptive Task Scheduling in Fog Computing

An Overview of Hadoop Job Scheduling Algorithms for Big Data

Contact Info

Product

Resources

About