Performance prediction from simulation systems to physical systems using machine learning with transfer learning and scaling

Abstract

Summary Selection from several computer systems with different hardware features resulting in different software performance is a critical problem to solve. The problem becomes even more challenging when access to computer systems with different features is difficult. We had proposed a novel solution, “cross performance prediction with scaling,” in our previous work. In the scaling model, we predicted the physical system's runtime using a machine learning model trained only on a performance dataset of simulation‐based systems applying a scaling factor to the predicted runtime. In this article, we propose another novel idea, “cross performance prediction with transfer learning,” that uses transfer learning to solve the same problem. This model predicts the target physical system's performance using a machine learning model trained on a combined performance dataset from simulation‐based systems and an accessible source physical system. We evaluate both the models using several benchmark algorithms from SD‐VBS and MiBench suites. Our scaling model results have achieved a prediction error of 10%–25% for general‐purpose systems, whereas the transfer learning model has higher errors in the range of 50%. We have also developed a method to extract the rules built during the decision tree model's training to predict the runtime.