Combining features of negative correlation learning with mixture of experts in proposed ensemble methods

Masoudnia, Saeed; Ebrahimpour, Reza; Arani, Seyed Ali Asghar Abbaszadeh

doi:10.1016/j.asoc.2012.07.022

Cited by 17 publications

(11 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…One approach to deal with complex, real-world problems is to combine AI prediction models and form an ensemble of predictors that exploit the different local behaviors of the base models to improve the overall prediction system's performance (Masoudnia et al, 2012). The main objective of ensemble learning methods is to simplify a difficult prediction task by dividing it into some relatively easy prediction subtasks and formulating a consensus prediction result for the original data (García-Pedrajas et al, 2012).…”

Section: Introduction and Literature Reviewmentioning

confidence: 99%

“…The key idea behind NCL is to introduce a correlation penalty term to the cost function of individual NN components so that each component minimizes its mean square error (MSE) together with the ensemble's error correlation (Masoudnia et al, 2012). Alhamdoosh and Wang (2014) incorporated random vector functional link (RVFL) networks as base components with the NCL strategy to build neural network ensembles.…”

Section: Introduction and Literature Reviewmentioning

confidence: 99%

See 1 more Smart Citation

A novel Boosted-neural network ensemble for modeling multi-target regression problems

Hadavandi

Shahrabi

Shamshirband

2015

Engineering Applications of Artificial Intelligence

View full text Add to dashboard Cite

Section: Introduction and Literature Reviewmentioning

confidence: 99%

Section: Introduction and Literature Reviewmentioning

confidence: 99%

A novel Boosted-neural network ensemble for modeling multi-target regression problems

Hadavandi

Shahrabi

Shamshirband

2015

Engineering Applications of Artificial Intelligence

View full text Add to dashboard Cite

“…In other words, MI provides a decision-making system that can be used to select the most effective inputs (variables which can represent the influence of other ones) and reduce the noises for the development of a model (Chelgani et al 2018). In a predictive modeling problem, various researches indicated that combination of intelligent predictor models and development of an ensemble of predictors (experts) can construct an accurate model to deal with complicated problems (Masoudnia et al 2012;Hadavandi et al 2015Hadavandi et al , 2016. One of the popular ensemble methods is the neural network ensemble (NNE) (Hansen and Salamon 1990) and an efficient approach for creating an NNE model is Adaptive Boosting (Adaboost) that can adaptively improve the probability of sampling cases for accurate training experts for the NNE model.…”

Section: Introductionmentioning

confidence: 99%

“…One of the popular ensemble methods is the neural network ensemble (NNE) (Hansen and Salamon 1990) and an efficient approach for creating an NNE model is Adaptive Boosting (Adaboost) that can adaptively improve the probability of sampling cases for accurate training experts for the NNE model. This approach can develop a model by using a wide distribution of inputs and reduce the prediction errors by considering the information of previous experts (Hansen and Salamon 1990;Freund and Schapire 1996;Solomatine and Shrestha 2004;Masoudnia et al 2012;Tian et al 2012;Zhai et al 2012). Although the last decade has witnessed increasing applications of MI and Adaboost-NNE models, they have not yet been used in the exploration and prediction of earth science sectors.…”

Section: Introductionmentioning

confidence: 99%

Exploring relationships between mechanical properties of marl core samples by a coupling of mutual information and predictive ensemble model

Salehin

Hadavandi

Chelgani

2019

Model. Earth Syst. Environ.

View full text Add to dashboard Cite

Inappropriate evaluation of uniaxial compression indexes (E and UCS) of rocks in high seismic intensity areas such as dam regions can lead to underestimation of the load, and possible settlement of the structure. Indirect assessments of these rock mechanical indexes based on non-destructive experiments and by using intelligent models is a well-accepted method to overcome associated limitations with laboratory tests of E and UCS. This study introduces the mutual information (MI) method as a unique system for variable importance measurement (VIM) and feature selection. Conducting MI-VIM assessments between various analyses of marl core samples (depth, density, ultrasonic tests (ν d , V p and V s), Brazilian test (σt), triaxial compression test (C and and ϕ) and point load test (I s (50)) indicated that V s and σt had the highest importance for E and UCS prediction. adaptive boosting-neural network ensemble (Adaboost-NNE) was used for the prediction of E and UCS. Testing of the generated Adaboost-NNE indicated that this model could accurately predict UCS and E with correlations of determinations 0.98 and 0.92, respectively. These results showed that VIM of MI coupled with Adaboost-NNE could develop a robust model that can be used for the prediction and modeling of other indexes of rocks.

show abstract

“…This advantage significantly reduces the training cost of producing the ensemble. However, this method does not guarantee enough diversity between different snapshot networks while diversity is key for the success of ensemble(Brown et al, 2005;Masoudnia, Ebrahimpour, & Arani, 2012a). The authors claimed that this method could visit multiple good and diverse local minimums, leading to increasingly accurate predictions over the course of training in several classification benchmarks.…”

mentioning

confidence: 99%

Multi-representational learning for Offline Signature Verification using Multi-Loss Snapshot Ensemble of CNNs

Masoudnia

Mersa

Araabi

et al. 2019

Expert Systems with Applications

Self Cite

View full text Add to dashboard Cite

Offline Signature Verification (OSV) is a challenging pattern recognition task, especially in the presence of skilled forgeries that are not available during training. This study aims to tackle its challenges and meet the substantial need for generalization for OSV by examining different loss functions for Convolutional Neural Network (CNN). We adopt our new approach to OSV by asking two questions: 1. which classification loss provides more generalization for feature learning in OSV? , and 2. How integration of different losses into a unified multi-loss function lead to an improved learning framework?These questions are studied based on analysis of three loss functions, including cross entropy, Cauchy-Schwarz divergence, and hinge loss. According to complementary features of these losses, we combine them into a dynamic multi-loss function and propose a novel ensemble framework for simultaneous use of them in CNN. Our proposed Multi-Loss Snapshot Ensemble (MLSE) consists of several sequential trials. In each trial, a dominant loss function is selected from the multi-loss set, and the remaining losses act as a regularizer. Different trials learn diverse representations for each input based on signature identification task. This multi-representation set is then employed for the verification task. An ensemble of SVMs is trained on these representations, and their decisions are finally combined according to the selection of most generalizable SVM for each user.We conducted two sets of experiments based on two different protocols of OSV, i.e., writer-dependent and writer-independent on three signature datasets: GPDS-Synthetic, MCYT, and UT-SIG. Based on the writerdependent OSV protocol, On UT-SIG, we achieved 6.17% Equal Error Rate (EER) which showed substantial improvement over the best EER in the literature, 9.61%. Our method surpassed state-of-the-arts by 2.5% on GPDS-Synthetic, achieving 6.13%. Our result on MCYT was also comparable to the best previous results.The second set of experiments examined the robustness of our proposed method to the arrival of new users enrolled in the OSV system based on the writer-independent protocol. The results also confirmed that our proposed system efficiently performed the verification of new users enrolled in the OSV system.

show abstract

Combining features of negative correlation learning with mixture of experts in proposed ensemble methods

Cited by 17 publications

References 31 publications

A novel Boosted-neural network ensemble for modeling multi-target regression problems

A novel Boosted-neural network ensemble for modeling multi-target regression problems

Exploring relationships between mechanical properties of marl core samples by a coupling of mutual information and predictive ensemble model

Multi-representational learning for Offline Signature Verification using Multi-Loss Snapshot Ensemble of CNNs

Contact Info

Product

Resources

About