Comparative study of random search hyper-parameter tuning for software effort estimation

Villalobos-Arias, Leonardo; Quesada-López, Christian

doi:10.1145/3475960.3475986

Cited by 12 publications

(5 citation statements)

References 41 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…GS is a simple and easy method [21]. This method is most often used by researchers to perform optimization and is proven to improve accuracy [22][23][24][25][26][27][28][29][30][31][32][33]. GS is a technique for searching the search space for all possible combinations of the hyperparameters given by the user.…”

Section: Hyperparameters Tuning Techniquementioning

confidence: 99%

Optimization of Hyperparameters in Machine Learning for Enhancing Predictions of Student Academic Performance

Arifin¹,

Widowati²,

Farikhin³

2023

ISI

View full text Add to dashboard Cite

The prediction of student academic performance with high accuracy is of paramount importance in improving educational outcomes and developing tailored learning methodologies. It also serves as a preventative measure against student dropouts. This study is centered on enhancing the precision of such predictions by optimizing hyperparameters in machine learning techniques. In pursuit of optimal performance, a range of machine learning techniques is compared, and the most accurate one selected for hyperparameter optimization. The adopted method for this optimization is the Grid Search (GS) technique. It is found that hyperparameter optimization in the Gradient Boosting Regression Tree (GBRT) using the GS method bolsters the accuracy of predictions pertaining to student academic performance. The results obtained in this study are validated using a five-fold cross-validation method. This rigorous validation ensures the robustness of our findings. Thus, the study presents a critical contribution to the effective prediction of student academic performance, potentially informing the development of more efficient and personalized educational strategies.

show abstract

Section: Hyperparameters Tuning Techniquementioning

confidence: 99%

Optimization of Hyperparameters in Machine Learning for Enhancing Predictions of Student Academic Performance

Arifin¹,

Widowati²,

Farikhin³

2023

ISI

View full text Add to dashboard Cite

show abstract

“…These algorithms were chosen based on their accurate performance in the literatur for predicting annual building energy consumption, effectiveness in handling both cate gorical and numerical input features [17], and their robustness in non-linear relationship modelling [16,18]. The following section will provide a highlight of the key features and algorithms employed in the selected ML models.…”

Section: Model Selectionmentioning

confidence: 99%

“…In some cases, particularly when dealing with large and complex datasets, methods like random search can offer similar benefits to more sophisticated hyperparameter tuners while requiring fewer computational resources and being easier to implement. Empirical experiments have shown that a simple random search algorithm, sampling as few as 60 hyperparameter combinations, can perform as effectively as an exhaustive grid search spanning over 4000 hyperparameter values [18,30]. As a result, considering the complexity of the dataset, the randomized search method has been utilized in this study for the hyperparameter tuning process.…”

Section: Hyperparameter Tuningmentioning

confidence: 99%

Developing a Data-Driven AI Model to Enhance Energy Efficiency in UK Residential Buildings

Seraj,

Bahadori-Jahromi,

Amirkhani

2024

Sustainability

View full text Add to dashboard Cite

Residential buildings contribute 30% of the UK’s total final energy consumption. However, with less than one percent of its housing stock being replaced annually, retrofitting existing homes has significant importance in meeting energy-efficiency targets. Consequently, many physics-based and data-driven models and tools have been developed to analyse the effects of retrofit strategies from various points of view. This paper aims to develop a data-driven AI model that predicts buildings’ energy performance based on their features under various retrofit scenarios. In this context, four different machine learning models were developed based on the EPC (Energy Performance Certificate) dataset for residential buildings and standard assessment procedure (SAP) guidelines in the UK. Additionally, an interface was designed that enables users to analyse the effect of different retrofit strategies on a building’s energy performance using the developed AI models. The results of this study revealed the artificial neural network as the most accurate predictive model, with a coefficient of determination (R2) of 0.82 and a mean percentage error of 11.9 percent. However, some conceptual irregularities were observed across all the models when dealing with different retrofit scenarios. All summary, such tools can be further improved to offer a potential alternative or support to physics-based models, enhancing the efficiency of retrofitting processes in buildings.

show abstract

“…Since they cannot be learnt from the data, they are usually denoted as hyperparameters and provided explicitly. There are various strategies for choosing the best possible values of these hyperparameters, like grid search, random selection, simulated annealing, and other methods [47][48][49].…”

Section: Hyperparameter Tuning Gridsmentioning

confidence: 99%

The Impact of Data Quality on Software Testing Effort Prediction

Radliński

2023

Electronics

View full text Add to dashboard Cite

Background: This paper investigates the impact of data quality on the performance of models predicting effort on software testing. Data quality was reflected by training data filtering strategies (data variants) covering combinations of Data Quality Rating, UFP Rating, and a threshold of valid cases. Methods: The experiment used the ISBSG dataset and 16 machine learning models. A process of three-fold cross-validation repeated 20 times was used to train and evaluate each model with each data variant. Model performance was assessed using absolute errors of prediction. A ‘win–tie–loss’ procedure, based on the Wilcoxon signed-rank test, was applied to identify the best models and data variants. Results: Most models, especially the most accurate, performed the best on a complete dataset, even though it contained cases with low data ratings. The detailed results include the rankings of the following: (1) models for particular data variants, (2) data variants for particular models, and (3) the best-performing combinations of models and data variants. Conclusions: Arbitrary and restrictive data selection to only projects with Data Quality Rating and UFP Rating of ‘A’ or ‘B’, commonly used in the literature, does not seem justified. It is recommended not to exclude cases with low data ratings to achieve better accuracy of most predictive models for testing effort prediction.

show abstract

Comparative study of random search hyper-parameter tuning for software effort estimation

Cited by 12 publications

References 41 publications

Optimization of Hyperparameters in Machine Learning for Enhancing Predictions of Student Academic Performance

Optimization of Hyperparameters in Machine Learning for Enhancing Predictions of Student Academic Performance

Developing a Data-Driven AI Model to Enhance Energy Efficiency in UK Residential Buildings

The Impact of Data Quality on Software Testing Effort Prediction

Contact Info

Product

Resources

About