Investigation the Effect of Particle Swarm Optimization in Performance of Mixture of Experts

Abstract

Mixture of experts (ME) is one of the most popular and interesting combining methods, which has great potential to improve performance in machine learning. ME is established based on the divide-and-conquer principle in which the problem space is divided between a few neural network experts, supervised by a gating network. In earlier works on ME, different strategies were developed to divide the problem space between the experts. As result, we have introduced a new method based on the principles of Particle Swarm Optimization (PSO) as a learning step in ME. In this paper, different aspects of the proposed method are compared with the common version of ME. The result carried out from this paper shows that the new method is robust to the variation of ensemble complexity in terms of the number of individual experts, and the number of hidden units.