Evaluating Machine Learning Models for Disparate Computer Systems Performance Prediction

Mankodi, Amit; Bhatt, Amit; Chaudhury, Bhaskar; Kumar, Rajat; Amrutiya, Aditya

doi:10.1109/conecct50063.2020.9198512

Cited by 3 publications

(1 citation statement)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Decision tree regression is one of the popular machine learning models due to its simplicity and intelligibility. 21,22 Furthermore, in a separate study, we have performed, 23 we have shown that decision tree regression has higher prediction accuracy than other machine learning models for our performance dataset. Therefore, we have used scikit-learn, 24 a python-based library implementation of decision tree regression 25,26 as our machine learning model for our cross performance prediction.…”

Section: Decision Tree Regression Machine Learning Modelmentioning

confidence: 89%

Performance prediction from simulation systems to physical systems using machine learning with transfer learning and scaling

Mankodi

Bhatt

Chaudhury

2021

Concurrency and Computation

Self Cite

View full text Add to dashboard Cite

Summary Selection from several computer systems with different hardware features resulting in different software performance is a critical problem to solve. The problem becomes even more challenging when access to computer systems with different features is difficult. We had proposed a novel solution, “cross performance prediction with scaling,” in our previous work. In the scaling model, we predicted the physical system's runtime using a machine learning model trained only on a performance dataset of simulation‐based systems applying a scaling factor to the predicted runtime. In this article, we propose another novel idea, “cross performance prediction with transfer learning,” that uses transfer learning to solve the same problem. This model predicts the target physical system's performance using a machine learning model trained on a combined performance dataset from simulation‐based systems and an accessible source physical system. We evaluate both the models using several benchmark algorithms from SD‐VBS and MiBench suites. Our scaling model results have achieved a prediction error of 10%–25% for general‐purpose systems, whereas the transfer learning model has higher errors in the range of 50%. We have also developed a method to extract the rules built during the decision tree model's training to predict the runtime.

show abstract