A Weighted Ensemble Learning Algorithm Based on Diversity Using a Novel Particle Swarm Optimization Approach

Abstract

In ensemble learning, accuracy and diversity are the main factors affecting its performance. In previous studies, diversity was regarded only as a regularization term, which does not sufficiently indicate that diversity should implicitly be treated as an accuracy factor. In this study, a two-stage weighted ensemble learning method using the particle swarm optimization (PSO) algorithm is proposed to balance the diversity and accuracy in ensemble learning. The first stage is to enhance the diversity of the individual learner, which can be achieved by manipulating the datasets and the input features via a mixed-binary PSO algorithm to search for a set of individual learners with appropriate diversity. The purpose of the second stage is to improve the accuracy of the ensemble classifier using a weighted ensemble method that considers both diversity and accuracy. The set of weighted classifier ensembles is obtained by optimization via the PSO algorithm. The experimental results on 30 UCI datasets demonstrate that the proposed algorithm outperforms other state-of-the-art baselines.

Adaptive Particle Swarm Optimization Based Ensemble Classifier for Network Intrusion Detection

“…The vectors 𝑟 1 and 𝑟 2 adds stochastic properties to the algorithm. A large value of 𝜔 enables the optimization algorithm to get out of the local optimal solution and to enhance the global search ability whereas a smaller value of 𝜔 facilitates algorithm convergence and improves the local search ability [23]. The inertia weight is used to adjust the entire search process of the PSO algorithm.…”

Solving Cold‐Standby Reliability‐Redundancy Allocation Problems with Particle‐Based Simplified Swarm Optimization

“…P l−1 gBest is the global 4 Journal of Sensors best, which is the best solution of all existing solutions among all pBests; there is only one gBest at a time. In the l th generation, each solution Y l i moves towards pBest P l−1 i and gBest P l−1 gBest [19,20] for l = 0,1,2, ⋯, N g and i = 1, 2, ⋯, N so . The velocities and positions are updated according to the following equation after both P l−1 i and P l−1 gBest are found.…”

“…In this section, the PSSO is used together with an allvariable-UM (here termed n-UM and λ-UM) that retains the merits of PSO and SSO that are beneficial in searching for the optima in real and discrete numbers, respectively [15,[17][18][19][20][32][33][34][35][36][37][38][39]. This study considers how the simultaneous application of those merits is conducive to computing the RRAP with cold-standby strategy, which is a mixedinteger programming model.…”

“…Particle swarm optimization (PSO) that was first developed by Kennedy and Eberhard in 1995 [17] and simplified swarm optimization (SSO) that was originally exploited by Yeh in 2009 [18] are two of the most well-known algorithms in SI. In recent years, we have seen an increasing interest both in PSO [3,6,15,[19][20][21][22][23] and in SSO [12,13,15,[24][25][26][27][28] for solving larger problems in science and technology.…”