Backfilling Using System-Generated Predictions Rather than User Runtime Estimates

Tsafrir, Dan; Etsion, Yoav; Feitelson, Dror G.

doi:10.1109/tpds.2007.70606

Cited by 299 publications

(225 citation statements)

References 28 publications

Supporting

Mentioning

222

Contrasting

Order By: Relevance

“…To compare the results, we build an accuracy index for each reconstruction algorithm, each graph, and each studied network characteristic. For latencies and bandwidths, following [25], we define accuracy as the maximum of the two ratios x R /x M and x M /x R , where x R is the reconstructed value and x M is the original measured one. We compute the accuracy of each pair of nodes, and then the geometric mean of all accuracies.…”

Section: Methodsmentioning

confidence: 99%

A First Step Towards Automatically Building Network Representations

Eyraud-Dubois

Legrand

Quinson

et al. 2007

Euro-Par 2007 Parallel Processing

View full text Add to dashboard Cite

To fully harness Grids, users or middlewares must have some knowledge on the topology of the platform interconnection network. As such knowledge is usually not available, one must uses tools which automatically build a topological network model through some measurements. In this article, we define a methodology to assess the quality of these network model building tools, and we apply this methodology to representatives of the main classes of model builders and to two new algorithms. We show that none of the main existing techniques build models that enable to accurately predict the running time of simple application kernels for actual platforms. However some of the new algorithms we propose give excellent results in a wide range of situations. Key-words: Network model, topology reconstruction, GridsThis text is also available as a research report of the Laboratoire de l'Informatique du Parallélisme http://www.ens-lyon.fr/LIP. Une premièreétape vers la construction automatique de modèles de réseau d'interconnexionRésumé : Afin de tirer le meilleur parti des grilles, les utilisateurs et les intergiciels doivent avoir connaissance de la topologie du réseau d'interconnexion de la plate-forme utilisée. Comme cette connaissance n'est généralement pas disponible a priori, on doit avoir recours a des outils construisant un modèle du réseau d'interconnexionà partir de mesures. Dans cet article nous définissons une méthodologie pourévaluer la qualité de ces outils de construction de modèles de réseau, et nous l'appliquonsà des représentants des principaux types de reconstructeurs de topologies, ainsi qu'à deux nouveaux algorithmes. Nous montrons qu'aucune des techniques existantes ne produit des modèles qui permettent de prédire avec précision le temps d'exécution sur les plates-formes actuelles de simples noyaux d'applications. Au contraire, un des nouveaux algorithmes obtient de très bons résultats dans des situations très variées.

show abstract

Section: Methodsmentioning

confidence: 99%

A First Step Towards Automatically Building Network Representations

Eyraud-Dubois

Legrand

Quinson

et al. 2007

Euro-Par 2007 Parallel Processing

View full text Add to dashboard Cite

show abstract

“…This justifies the common assumption that recent behavior is indicative of future behavior, and can be exploited to our advantage. For example, we may be able to make predictions about resource utilization and availability, as is done in the Network Weather Service [737], and when predicting queueing times [87], job runtimes [694], and user clicks on web search results [550].…”

Section: Importancementioning

confidence: 99%

“…Thus a model based on random sampling from a global distribution subjects the system to a very different workload than one based on localized sampling from the same distribution. In terms of system behavior it is possible to envision schedulers that exploit the short-range regularity of localized workloads, and adapt their behavior to best suit the current workload [243,754,668,755,694]. This is similar to recent approaches used in memory allocators, which cache freed blocks in anticipation of future requests for the same block sizes [732].…”

Section: Importancementioning

confidence: 99%

“…This led to a line of interesting experimental work that eventually showed that inaccurate estimates cause holes to be left in the schedule, and these can then be filled with short jobs, effectively leading to an implicit shortest-job-first effect that improves the average response time. Thus the truth of the matter is that accurate estimates are actually better, because they facilitate using shortest-job-first explicitly [242,506,763,696,694]. The original study was also tainted by using "statistically bad" estimates that, as noted earlier, still contain significant information.…”

mentioning

confidence: 99%

See 1 more Smart Citation

Workload Modeling for Computer Systems Performance Evaluation

Feitelson

2015

Self Cite

191

131

View full text Add to dashboard Cite

Reliable performance evaluations require the use of representative workloads. This is no easy task since modern computer systems and their workloads are complex, with many interrelated attributes and complicated structures. Experts often use sophisticated mathematics to analyze and describe workload models, making these models difficult for practitioners to grasp. This book aims to close this gap by emphasizing the intuition and the reasoning behind the definitions and derivations related to the workload models. It provides numerous examples from real production systems, with hundreds of graphs. Using this book, readers will be able to analyze collected workload data and clean it if necessary, derive statistical models that include skewed marginal distributions and correlations, and consider the need for generative models and feedback from the system. The descriptive statistics techniques covered are also useful for other domains.

show abstract

“…Job scheduling and assignment problems are in general NP-hard [18]. In the case of online job scheduling, fast heuristic algorithms and policies are extensively used, such as firstcome first-serve (FCFS) augmented with back-filling [19]. With respect to energy-sustainability, previous research has focused on: (i) including economical models for job schedules [20]; (ii) avoiding or even preventing excessive heat conditions in data centers through job assignment algorithms [3,21,22] thus improving the sustainability (through reduction in cooling power requirement); and (iii) performing spatio-temporal job scheduling (i.e.…”

Section: Energy Perspectivementioning

confidence: 99%