Using Machine Learning Ensemble Methods to Predict Execution Time of e-Science Workflows in Heterogeneous Distributed Systems

Abstract

Effective planning and optimized execution of the e-Science workflows in distributed systems, such as the Grid, need predictions of execution times of the workflows. However, predicting the execution times of e-Science workflows in heterogeneous distributed systems is a challenging job due to the complex structure of workflows, variations due to input problem-sizes, and heterogeneous and dynamic nature of the shared resources. To this end, we propose two novel workflow execution time-prediction methods based on the machine learning ensemble models. In this paper, we showcase our approach for two different real Grid environments. Our approach can effectively predict the execution time of the scientific workflow applications in the Grid for various problem sizes, Grid sites, and runtime environments. We characterized the workflow performance in the Grid using the attributes that define structure of workflow as well as the execution environment. Contrary to common ensembles, our ensemble systems employed three strong learners, which balance the weaknesses of each other by their strengths to model the workflow execution times. The proposed methods have been thoroughly evaluated for three real-world e-science workflow applications. The experimental results demonstrated that our proposed multi-model ensemble models can significantly decrease the prediction error (by 50%, on average) as compared with methods based on the radial basis function neural network, local learning, and performance templates. The proposed methods can also be applied with similar effectiveness and without any major modification for other heterogeneous distributed environments, such as the Cloud. INDEX TERMS Machine learning ensemble systems, performance modeling of e-science workflows, distributed execution of e-science workflows, execution time prediction of e-science workflows.

A Dynamic Predictor Selection Method Based on Recent Temporal Windows for Time Series Forecasting

“…Dynamic selection approaches [9], [12], [13], [17] for time series forecasting have been proposed aiming to improve the accuracy of MPS. These approaches are based on [23] that seminally proposed a dynamic selection approach for regression tasks named herein to as Dynamic Selection by Local Accuracy (DS-LA).…”

“…For each w t of the out-of-sample, two regions of competence ∆ LT t and ∆ P R t are generated. Each region of competence is composed of 10 windows (k = 10), where ∆ LT t = (w m1 , w n2 , ..., w ik ) represent the windows from in-sample selected by the literature assumption [9], [12], [13], [17] (using Euclidean distance), and ∆ P R t = (w t−1 , ..., w t−k ) are the k closest windows to w t selected by the proposed assumption.…”

“…Dynamic selection approaches commonly use the performance of the models in the Region of Competence (RoC) as a criterion to select the most competent ones [9], [12], [13], [17]- [19]. The RoC is composed of the k patterns in the in-sample (training or validation sets) [12], which are more similar to the test pattern according to some measure such as the Euclidean distance [20].…”

CLUTCH: A Clustering-Driven Runtime Estimation Scheme for Scientific Simulations

“…Table 1 summarizes major similarities and differences between our work and the existing studies. First, the examined papers concern various application domains: load sharing facility (LSF) [8], parallel program [9], [11], [25], cloud [10], [29], HPC [2], [14], [15], [19], location-based services [20]- [23], databases [26]- [28], big data applications [29], [30], and scientific workloads [12], [13], [16]- [19]. The runtime estimation problem addressed in this paper applies to the scientific workloads domain.…”

“…In this paper we present a novel "ensemble" of clustering, regression, and classification to construct a runtime estimation model. Unlike previous works [13], [18] which take a similar approach, the biggest difference in our work is to apply clustering to the input data in the early training process, resulting in estimation quality enhancement.…”