Accelerating Batch Analytics with Residual Resources from Interactive Clouds

Clay, R. Benjamin; Shen, Zhiming; Ma, Xiaosong

doi:10.1109/mascots.2013.63

Cited by 9 publications

(9 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The performance disadvantages of public clouds for parallel and scientific computing in comparison to grids and other parallel computing infrastructures have been documented in [17]. Optimizing cluster sizes across a range of workloads and goals via tools that can leverage residual or unused resources due to overprovisioning is proposed by [18]. Zhang et al designed an evaluation framework that focuses on evaluating and selecting of different available underlying cloud computing platforms (e.g.…”

Section: Related Workmentioning

confidence: 99%

Less Can Be More: Micro-managing VMs in Amazon EC2

Wen

Lü

Casale

et al. 2015

2015 IEEE 8th International Conference on Cloud Computing

View full text Add to dashboard Cite

Abstract-Micro instances (t1.micro) are the class of Amazon EC2 virtual machines (VMs) offering the lowest operational costs for applications with short bursts in their CPU requirements. As processing proceeds, EC2 throttles CPU capacity of micro instances in a complex, unpredictable, manner. This paper aims at making micro instances more predictable and efficient to use. First, we present a characterization of EC2 micro instances that evaluates the complex interactions between cost, performance, idleness and CPU throttling. Next, we define adaptive algorithms to manage CPU consumption by learning the workload characteristics at runtime and by injecting idleness to diminish hostlevel throttling. We show that a gradient-hill strategy leads to favorable results. For CPU bound workloads, we observe that a significant portion of jobs (up to 65%) can have end-to-end times that are even four times shorter than those of the more expensive m1.small class. Our algorithms drastically reduce the long tails of job execution times on the micro instances, resulting to favorable comparisons against even small instances.

show abstract

Section: Related Workmentioning

confidence: 99%

Less Can Be More: Micro-managing VMs in Amazon EC2

Wen

Lü

Casale

et al. 2015

2015 IEEE 8th International Conference on Cloud Computing

View full text Add to dashboard Cite

show abstract

“…Previous works [5], [9], [12], [14] have shown heterogeneous cluster designs wherein a core set of dedicated nodes running batch jobs are complemented by residual resources from volunteer nodes, or in some cases using "spot instances" from EC2 [4]. The closest work to ours is by Clay et al [5].…”

Section: Related Workmentioning

confidence: 93%

“…This problem was first tackled by Clay et al, who described how to pick the appropriate number of shared nodes in order to maximize performance and minimize overall energy costs [5]. Like their work, we focus on scheduling and modeling the Map phase since this is generally the larger portion of the program, and is less prone to performance problems due to slow nodes.…”

Section: Background and Motivation A Map Reduce In Virtualized Cmentioning

confidence: 99%

“…If too many I/O intensive tasks are run simultaneously, task completion times may begin to rise [5]. To prevent this, we use a model to predict the I/O load incurred by starting a new map task.…”

Section: Map Phase Completion Timementioning

confidence: 99%

“…It finds the job with the smallest normT CT value, since that job is best matched for the available resources. • Before assigning the slot, MP calculates the IO cost of running the selected job using Equation 5. If starting a new task of this type will cause any of the data nodes to become overloaded, then the job with the next highest normT CT is considered, and so on.…”

Section: B Progress Aware Earliest Deadline Firstmentioning

confidence: 99%

See 2 more Smart Citations

MIMP: Deadline and Interference Aware Scheduling of Hadoop Virtual Machines

Zhang

Rajasekaran

Wood

et al. 2014

2014 14th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing

View full text Add to dashboard Cite

Virtualization promised to dramatically increase server utilization levels, yet many data centers are still only lightly loaded. In some ways, big data applications are an ideal fit for using this residual capacity to perform meaningful work, but the high level of interference between interactive and batch processing workloads currently prevents this from being a practical solution in virtualized environments. Further, the variable nature of spare capacity may make it difficult to meet big data application deadlines.In this work we propose two schedulers: one in the virtualization layer designed to minimize interference on high priority interactive services, and one in the Hadoop framework that helps batch processing jobs meet their own performance deadlines. Our approach uses performance models to match Hadoop tasks to the servers that will benefit them the most, and deadline-aware scheduling to effectively order incoming jobs. The combination of these schedulers allows data center administrators to safely mix resource intensive Hadoop jobs with latency sensitive web applications, and still achieve predictable performance for both. We have implemented our system using Xen and Hadoop, and our evaluation shows that our schedulers allow a mixed cluster to reduce web response times by more than ten fold, while meeting more Hadoop deadlines and lowering total task execution times by 6.5%.

show abstract

Design science research contribution to business intelligence in the cloud — A systematic literature review

Mwilu

Comyn-Wattiau

Prat

2016

Future Generation Computer Systems

View full text Add to dashboard Cite

Business intelligence (BI) helps managers make informed decisions. In the age of big data, BI technology provides essential support for decision making. Cloud computing also attracts many organizations because of its potential: ubiquitous, convenient, on-demand network access to a shared pool of configurable computing resources (e.g. networks, servers, storage, applications, and services). This paper focuses on the deployment of BI in the cloud, from the vantage point of design science research (DSR). We produce a state of the art of research pertaining to BI in the cloud, following the methodology of systematic literature review. This literature review especially exhibits the different artifacts proposed by design science researchers regarding BI in the cloud. To structure the literature review, we propose a framework composed of two dimensions: artifact type and BI step. In particular, we propose a typology of artifact types, refining the coarse-grained typology commonly used in DSR. We use the two-dimensional framework both to map the current state of DSR regarding BI in the cloud, and to elicit future research avenues in terms of design science artifacts for BI in the cloud. The contribution is threefold: the literature review may help DSR researchers get an overview of this active research domain; the two-dimensional framework facilitates the understanding of different research streams; finally, the proposed future topics may guide researchers in identifying promising research avenues.

show abstract

Accelerating Batch Analytics with Residual Resources from Interactive Clouds

Cited by 9 publications

References 8 publications

Less Can Be More: Micro-managing VMs in Amazon EC2

Less Can Be More: Micro-managing VMs in Amazon EC2

MIMP: Deadline and Interference Aware Scheduling of Hadoop Virtual Machines

Design science research contribution to business intelligence in the cloud — A systematic literature review

Contact Info

Product

Resources

About