Bayesian optimization based dynamic ensemble for time series forecasting

Du, Liang; Gao, Ruobin; Suganthan, Ponnuthurai Nagaratnam; Wang, David Z.W.

doi:10.1016/j.ins.2022.01.010

Cited by 110 publications

(23 citation statements)

References 37 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Hyperparameter tuning or optimization is a robust method of identifying and finding the best feasible values of hyperparameters for a machine learning model to attain the desired resultant modeling outcome. Popular hyperparameter tuning algorithms in the literature include random search, grid search, and Bayesian optimization search [ 36 , 37 ].…”

Section: Theoretical Backgroundmentioning

confidence: 99%

Machine Learning-Based Boosted Regression Ensemble Combined with Hyperparameter Tuning for Optimal Adaptive Learning

Isabona

Imoize

Kim

2022

Sensors

View full text Add to dashboard Cite

Over the past couple of decades, many telecommunication industries have passed through the different facets of the digital revolution by integrating artificial intelligence (AI) techniques into the way they run and define their processes. Relevant data acquisition, analysis, harnessing, and mining are now fully considered vital drivers for business growth in these industries. Machine learning, a subset of artificial intelligence (AI), can assist, particularly in learning patterns in big data chunks, intelligent extrapolative extraction of data and automatic decision-making in predictive learning. Firstly, in this paper, a detailed performance benchmarking of adaptive learning capacities of different key machine-learning-based regression models is provided for extrapolative analysis of throughput data acquired at the different user communication distances to the gNodeB transmitter in 5G new radio networks. Secondly, a random forest (RF)-based machine learning model combined with a least-squares boosting algorithm and Bayesian hyperparameter tuning method for further extrapolative analysis of the acquired throughput data is proposed. The proposed model is herein referred to as the RF-LS-BPT method. While the least-squares boosting algorithm is engaged to turn the possible RF weak learners to form stronger ones, resulting in a single strong prediction model, the Bayesian hyperparameter tuning automatically determines the best RF hyperparameter values, thereby enabling the proposed RF-LS-BPT model to obtain desired optimal prediction performance. The application of the proposed RF-LS-BPT method showed superior prediction accuracy over the ordinary random forest model and six other machine-learning-based regression models on the acquired throughput data. The coefficient of determination (Rsq) and mean absolute error (MAE) values obtained for the throughput prediction at different user locations using the proposed RF-LS-BPT method range from 0.9800 to 0.9999 and 0.42 to 4.24, respectively. The standard RF models attained 0.9644 to 0.9944 Rsq and 5.47 to 12.56 MAE values. The improved throughput prediction accuracy of the proposed RF-LS-BPT method demonstrates the significance of hyperparameter tuning/optimization in developing precise and reliable machine-learning-based regression models. The projected model would find valuable applications in throughput estimation and modeling in 5G and beyond 5G wireless communication systems.

show abstract

Section: Theoretical Backgroundmentioning

confidence: 99%

Machine Learning-Based Boosted Regression Ensemble Combined with Hyperparameter Tuning for Optimal Adaptive Learning

Isabona

Imoize

Kim

2022

Sensors

View full text Add to dashboard Cite

show abstract

“…Various ways of combining the weak learners can be used to create a model that has greater capacity, such as bagging, boosting, and stacking [52]. Further optimized models, such as the bayesian optimization-based dynamic ensemble proposed by Du et al [53] can be used, and are even applied with nonlinear data [54].…”

Section: Related Workmentioning

confidence: 99%

Wavelet Long Short-Term Memory to Fault Forecasting in Electrical Power Grids

Branco¹,

Cavalca²,

Stefenon³

et al. 2022

Preprint

View full text Add to dashboard Cite

The electric power distribution utility is responsible for providing energy to consumers in a continuous and stable way, failures in the electrical power system reduce the reliability indexes of the grid, directly harming its performance. For this reason, there is a need for failure prediction to reestablish power in the shortest possible time. Considering an evaluation of the number of failures over time, this paper proposes to perform a failure prediction during the first year of the pandemic in Brazil (2020) to verify the feasibility of using time series forecasting models for fault prediction. The Long Short-Term Memory (LSTM) model will be evaluated to obtain a forecast result that can be used by the electric power utility to organize the maintenance teams. The Wavelet transform shows to be promising in improving the predictive ability of the LSTM, making the Wavelet LSTM model suitable for the study at hand. The results show that the proposed approach has better results regarding the evaluation of the error in prediction and has robustness when a statistical analysis is performed.

show abstract

“…The acquisition function measures the utility of candidate points for the next evaluation by trading off the exploration of uncertain areas and exploiting promising regions. BO has been extensively used in many scenarios, including hyperparameter tuning [9,14,31,42,68], experimental design [22] and controller tuning [11,20,21,46]. Contextual BO considers the environmental conditions by augmenting the GP kernel with extra context variables and uses 𝐶𝐺𝑃 − 𝑈𝐶𝐵 to select promising action [35].…”

Section: Related Workmentioning

confidence: 99%

Towards Dynamic and Safe Configuration Tuning for Cloud Databases

Zhang,

Wu,

et al. 2022

Preprint

View full text Add to dashboard Cite

Configuration knobs of database systems are essential to achieve high throughput and low latency. Recently, automatic tuning systems using machine learning methods (ML) have shown to find better configurations compared to experienced database administrators (DBAs). However, there are still gaps to apply the existing systems in production environments, especially in the cloud. First, they conduct tuning for a given workload within a limited time window and ignore the dynamicity of workloads and data. Second, they rely on a copied instance and do not consider the availability of the database when sampling configurations, making the tuning expensive, delayed, and unsafe. To fill these gaps, we propose OnlineTune, which tunes the online databases safely in changing cloud environments. To accommodate the dynamicity, OnlineTune embeds the environmental factors as context feature and adopts contextual Bayesian Optimization with context space partition to optimize the database adaptively and scalably. To pursue safety during tuning, we leverage the black-box and the white-box knowledge to evaluate the safety of configurations and propose a safe exploration strategy via subspace adaptation.We conduct evaluations on dynamic workloads from benchmarks and real-world workloads. Compared with the state-of-the-art methods, OnlineTune achieves 14.4%~165.3% improvement on cumulative performance while reducing 91.0%~99.5% unsafe configuration recommendations. CCS CONCEPTS• Information systems → Autonomous database administration; • Computing methodologies → Machine learning.

show abstract

Bayesian optimization based dynamic ensemble for time series forecasting

Cited by 110 publications

References 37 publications

Machine Learning-Based Boosted Regression Ensemble Combined with Hyperparameter Tuning for Optimal Adaptive Learning

Machine Learning-Based Boosted Regression Ensemble Combined with Hyperparameter Tuning for Optimal Adaptive Learning

Wavelet Long Short-Term Memory to Fault Forecasting in Electrical Power Grids

Towards Dynamic and Safe Configuration Tuning for Cloud Databases

Contact Info

Product

Resources

About