Multi-objective job placement in clusters

Blagodurov, Sergey; Fedorova, Alexandra; Vinnik, Evgeny; Dwyer, Tyler; Hermenier, Fabien

doi:10.1145/2807591.2807636

Cited by 26 publications

(13 citation statements)

References 55 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Calling omp_set_dop blocks the caller until changing the DoP takes effect. 3 Communicating with the SCALO daemon. For the third extension, the parallel runtime creates a dedicated management thread to communicate with the SCALO daemon.…”

Section: Implementing Instrumentation and Adaptivity In The Openmp Rumentioning

confidence: 99%

“…A single, parallel program is often limited in how effectively it can use this increasing hardware parallelism. Co-locating jobs, that is co-executing multiple parallel programs, on a node can increase throughput and energy efficiency [3][4][5]27]. As the parallelism continues to increase, efficient execution of some workloads will require node sharing [27], and some systems already enable users to share a node by statically partitioning cores between different jobs.…”

mentioning

confidence: 99%

See 1 more Smart Citation

Scalo

Georgakoudis

Vandierendonck

Thoman

et al. 2017

ACM Trans. Archit. Code Optim.

View full text Add to dashboard Cite

Shared memory machines continue to increase in scale by adding more parallelism through additional cores and complex memory hierarchies. Often, executing multiple applications concurrently, dividing among them hardware threads, provides greater efficiency rather than executing a single application with large thread counts. However, contention for shared resources can limit the improvement of concurrent application execution: orchestrating the number of threads used by each application and is essential.In this article, we contribute SCALO, a solution to orchestrate concurrent application execution to increase throughput. SCALO monitors co-executing applications at runtime to evaluate their scalability. Its optimizing thread allocator analyzes these scalability estimates to adapt the parallelism of each program. Unlike previous approaches, SCALO differs by including dynamic contention effects on scalability and by controlling the parallelism during the execution of parallel regions. Thus, it improves throughput when other state-of-theart approaches fail and outperforms them by up to 40% when they succeed.

show abstract

Section: Implementing Instrumentation and Adaptivity In The Openmp Rumentioning

confidence: 99%

mentioning

confidence: 99%

Scalo

Georgakoudis

Vandierendonck

Thoman

et al. 2017

ACM Trans. Archit. Code Optim.

View full text Add to dashboard Cite

show abstract

“…Our approach is based on a multi-objective scheduling algorithm focusing on minimizing a weighted sum of objectives. The advantage of such approach is that it is automatically guided by predetermined weights while the disadvantage is that it is hard to determine the right values for the weights [10]. In contrast, a posteriori methods produce a Pareto front of solutions without predetermined values [10].…”

Section: Related Workmentioning

confidence: 99%

“…The advantage of such approach is that it is automatically guided by predetermined weights while the disadvantage is that it is hard to determine the right values for the weights [10]. In contrast, a posteriori methods produce a Pareto front of solutions without predetermined values [10]. Each solution is better than the others with respect to at least one objective and users can choose one from the produced solutions.…”

Section: Related Workmentioning

confidence: 99%

Multi-objective scheduling of Scientific Workflows in multisite clouds

Pacitti

Valduriez

et al. 2016

Future Generation Computer Systems

View full text Add to dashboard Cite

Clouds appear as appropriate infrastructures for executing Scientific Workflows (SWfs). A cloud is typically made of several sites (or data centers), each with its own resources and data. Thus, it becomes important to be able to execute some SWfs at more than one cloud site because of the geographical distribution of data or available resources among different cloud sites. Therefore, a major problem is how to execute a SWf in a multisite cloud, while reducing execution time and monetary costs. In this paper, we propose a general solution based on multi-objective scheduling in order to execute SWfs in a multisite cloud. The solution consists of a multi-objective cost model including execution time and monetary costs, a Single Site Virtual Machine (VM) Provisioning approach (SSVP) and ActGreedy, a multisite scheduling approach. We present an experimental evaluation, based on the execution of the SciEvol SWf in Microsoft Azure cloud. The results reveal that our scheduling approach significantly outperforms two adapted baseline algorithms (which we propose by adapting two existing algorithms) and the scheduling time is reasonable compared with genetic and brute-force algorithms. The results also show that our cost model is accurate and that SSVP can generate better VM provisioning plans compared with an existing approach.Work partially funded by EU H2020 Programme and MCTI/RNP-Brazil (HPC4E grant agreement number 689772), CNPq, FAPERJ, and INRIA (MUSIC project), Microsoft\ud (ZcloudFlow project) and performed in the context of the Computational Biology Institute (www.ibc-montpellier.fr). We would like to thank Kary Ocaña for her help in modeling and\ud executing the SciEvol SWf.Peer ReviewedPostprint (author's final draft

show abstract

“…Multi-objective optimization. We utilize the weighted sum approach to transform the multiobjective optimization problem to a single objective optimization problem, which has been widely used in the previous studies [120,74]. Denote the weights for resource efficiency, job latency and fairness are W e , W q and W f , respectively.…”

Section: Motivating Examplesmentioning

confidence: 99%

A study of big data computing platforms : performance, fairness and energy consumption

Niu¹

View full text Add to dashboard Cite

Today, population aging is a trend spreading across the world. It is estimated that the number of people aged 60 years or over, is increasing rapidly. Although the elderly has wisdom and wealth gathered from their life experience, they often require long-term assistance from others. Ambient assisted living (AAL) systems open up a new opportunity to address the needs of aged by utilizing information and communication techniques. Multiple sensors and actuators coexist in ambient assisted living systems and are deployed everywhere surrounding elderly. A huge volume of sensed data is collected continuously from elderly, storage of these massive raw data and efficient processing on them to infer knowledge from the data and also guide the actuators to meet the needs of elderly is a problem we face. In the thesis, we mainly focus on the efficient large-scale data processing, we investigate whether and how we can build an effective large-scale data processing system to effectively excavate knowledge, wisdom and skills from elderly people to impact the development of the entire society.A lot of large-scale big data analytics systems have thereby emerged to process the massive data effectively. Efficient job scheduling and resource management for these data analytic frameworks are nontrivial. Modern job schedulers and resource coordinators in data processing frameworks often need to consider multiple objectives simultaneously due to various system operators requirements on data analytics. Currently, resource efficiency (throughput), job latency (per-job performance), fairness (isolation guarantee) and energy consumption are important concerns for the job scheduler in modern large-scale multiple-tenant environments. Resource efficiency is de facto the very important factor for the big data analytic framework. Job latency reflects the waiting time of the application. Fairness is a key building block of any multiple-tenant computing system that allows resource sharing effectively. Energy usage of the data center has reached 3% of the global electricity consumption while generating 200 million metric tons of CO2 in 2014. In order to reduce carbon emission and financial burden on the electricity, a lot of data centers have been re-designed and powered with multiple energy sources, including renewable (green) energy from non-polluting sources and brown energy from traditional polluting sources.Improving the resource efficiency and reducing the per-job latency are the common sense on those large-scale data processing frameworks and fruitful studies are proposed in these directions. In i I would like to thank my friends and colleagues in IGS-LILY and PDSL. I will always remember all the great memories we created together in the part four years.I gratefully acknowledge the scholarship provided by LILY, IGS(Interdisciplinary Graduate School) and Nanyang Technological University for my doctorial program.Last but not least, I want to thank my family and friends for their unwavering support throughput my PhD.

show abstract

Multi-objective job placement in clusters

Cited by 26 publications

References 55 publications

Scalo

Scalo

Multi-objective scheduling of Scientific Workflows in multisite clouds

A study of big data computing platforms : performance, fairness and energy consumption

Contact Info

Product

Resources

About