Cost-Efficient Tasks and Data Co-Scheduling with AffordHadoop

Ehsan, Moussa; Chandrasekaran, Karthiek; Chen, Yao; Sion, Radu

doi:10.1109/tcc.2017.2702661

Cited by 10 publications

(14 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In [13], an Afford-Hadoop application is adopted to reduce cost in finishing various tasks and to allocate data and schedule tasks and hence efficiency of system get enhanced. However, a NP-hard problem occurs while scheduling different tasks in state-of-art technique.…”

Section: Related Workmentioning

confidence: 99%

“…In recent years, to enhance the performance of Hadoop application, various efficient Hadoop models are presented by numerous researchers [9][10][11][12][13][14]. A comprehensive information for job prediction and optimization can be presented using Starfish [9] model, which gathers an active Hadoop task profile at a satisfactory granularity.…”

Section: Introductionmentioning

confidence: 99%

“…However, gathering an active Hadoop task profile with comprehensive information can lead to large overhead and hence high over-predicted task run-time. In [11,13] utilizes both overlapping and non-overlapping phases of shuffle stage and for task prediction a conventional linear regression method is used. This application also helps to predict the required number of resources for various tasks with deadline constraints.…”

Section: Introductionmentioning

confidence: 99%

“…However, in CRESP application models, the effect of number of reduced jobs are discarded. In [13,14], the number of reduced jobs are constant. In CRESP model,  ISSN: 2088-8708 Int J Elec & Comp Eng, Vol.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Optimized memory model for hadoop map reduce framework

Bhaskar¹,

Ranjan

2019

IJECE

View full text Add to dashboard Cite

Map Reduce is the preferred computing framework used in large data analysis and processing applications. Hadoop is a widely used Map Reduce framework across different community due to its open source nature. Cloud service provider such as Microsoft azure HDInsight offers resources to its customer and only pays for their use. However, the critical challenges of cloud service provider is to meet user task Service level agreement (SLA) requirement (task deadline). Currently, the onus is on client to compute the amount of resource required to run a job on cloud. This work present a novel memory optimization model for Hadoop Map Reduce framework namely MOHMR (Optimized Hadoop Map Reduce) to process data in real-time and utilize system resource efficiently. The MOHMR present accurate model to compute job memory optimization and also present a model to provision the amount of cloud resource required to meet task deadline. The MOHMR first build a profile for each job and computes memory optimization time of job using greedy approach. Experiment are conducted on Microsoft Azure HDInsight cloud platform considering different application such as text computing and bioinformatics application to evaluate performance of MOHMR of over existing model shows significant performance improvement in terms of computation time. Experiment are conducted on Microsoft Azure HDInsight cloud. Overall, good correlation is reported between practical memory optimization values and theoretical memory optimization values.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Optimized memory model for hadoop map reduce framework

Bhaskar¹,

Ranjan

2019

IJECE

View full text Add to dashboard Cite

show abstract

“…Users can submit bids to the market at any time, using the spot price history to decide how much to bid. The cloud provider sets the spot price at regular time intervals, e.g., every five minutes, depending on the number of bids received from users (demand) and how many resources are available (supply) at each time slot [97] [101]. In these mechanisms, users' bids above the spot price are accepted, and that below is rejected in each time slot.…”

Section: Minimize Costmentioning

confidence: 99%

Review of the quality of service scheduling mechanisms in cloud

RemeshBabu¹,

Samuel²

2018

IJET

View full text Add to dashboard Cite

Cloud computing provides on demand access to a large pool of heterogeneous computational and storage resources to users over the internet. Optimal scheduling mechanisms are needed for the efficient management of these heterogeneous resources. The optimal scheduler can improve the Quality of Services (QoS) as well as maintaining efficiency and fairness among these tasks. In large scale distributed systems, the performance of these scheduling algorithms is crucial for better efficiency. Now the cloud customers are charged based upon the amount of resources they are consumed or held in reserve. Comparing these scheduling algorithms from different perspectives is needed for further improvement. This paper provides a comparative study about different resource allocation, load balancing and virtual machine consolidation algorithms in cloud computing. These algorithms have been evaluated in terms of their ability to provide QoS for the tasks and Service Level Agreement (SLA) guarantee amongst the jobs served. This study identifies current and future research directions in this area for QoS enabled cloud scheduling.

show abstract