SPECnet

Das, Dibyendu; Raghavendra, Prakash S.; Ramachandran, Arun

doi:10.1145/3185768.3186301

Companion of the 2018 ACM/SPEC International Conference on Performance Engineering 2018

DOI: 10.1145/3185768.3186301

|View full text |Cite

SPECnet

Dibyendu Das¹,

Prakash S. Raghavendra²,

Arun Ramachandran³

Help me understand this report

Search citation statements

Order By: Relevance

Paper Sections

Select...

Citation Types

Supporting

Mentioning

Contrasting

Year Published

2022

Publication Types

Select...

Article1

Relationship

Self Cite0

Independent1

Authors

Journals

Cited by 1 publication

References 3 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

Comparative Analysis of Machine Learning Models for Performance Prediction of the SPEC Benchmarks

Tousi

Luján

2022

IEEE Access

View full text Add to dashboard Cite

Simulation-based performance prediction is complicated and time-consuming. In this study, we apply supervised learning to predict the performance scores of Standard Performance Evaluation Corporation (SPEC) benchmarks. The SPEC CPU2017 is a public dataset of results obtained by executing 43 standardised performance benchmarks organised into 4 suites on various system configurations. This paper analyses the dataset and aims to answer the following questions: I) can we accurately predict the SPEC results based on the configurations provided in the dataset, without having to actually run the benchmarks? II) what are the most important hardware and software features? III) what are the best predictive models and hyperparameters, in terms of prediction error and time? and IV) can we predict the performance of future systems using the past data? We present how to prepare data, select features, tune hyperparameters and evaluate regression models based on Multi-Task Elastic-Net, Decision Tree, Random Forest, and Multi-Layer Perceptron neural networks estimators. Feature selection is performed in three steps: removing zero variance features, removing highly correlated features, and Recursive Feature Elimination based on different feature importance metrics: elastic-net coefficients, tree-based importance measures and Permutation Importance. We select the best models using grid search on the hyperparameter space, and finally, compare and evaluate the performance of the models. We show that tree-based models with the original 29 features provide accurate predictions with an average error of less than 4%. The average error of faster Decision Tree and Random Forest models with 10 features is still below 6% and 5% respectively.

show abstract

Comparative Analysis of Machine Learning Models for Performance Prediction of the SPEC Benchmarks

Tousi

Luján

2022

IEEE Access

View full text Add to dashboard Cite

show abstract

scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.

Contact Info

customersupport@researchsolutions.com

10624 S. Eastern Ave., Ste. A-614

Henderson, NV 89052, USA

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

Blog Terms and Conditions API Terms Privacy Policy Contact Cookie Preferences Do Not Sell or Share My Personal Information

Made with 💙 for researchers

Part of the Research Solutions Family.

SPECnet

Cited by 1 publication

References 3 publications

Comparative Analysis of Machine Learning Models for Performance Prediction of the SPEC Benchmarks

Comparative Analysis of Machine Learning Models for Performance Prediction of the SPEC Benchmarks

Contact Info

Product

Resources

About