Global optimization algorithms for training product unit neural networks

Ismail, Anis; Engelbrecht, Andries P.

doi:10.1109/ijcnn.2000.857826

Cited by 54 publications

(36 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…The Ý Ø is estimated differential coefficient of observed gene expression level at time Ø. The reason why we use equation (11) instead of (10) is to evaluate accurately when small amount of equation (10).…”

Section: The Inference Methods Using Punn Modelmentioning

confidence: 99%

“…Here, the gradient descent method, which is widely used for learning neural networks, fails to leaning PUNN in general [9], [10]. In this study, because of using the optimization algorithm to lean the PUNN , the minimized function (11) is not called SMC 2008 "error function" but called "the objective function".…”

Section: The Inference Methods Using Punn Modelmentioning

confidence: 99%

“…OPTIMIZATION METHOD The gradient descent method, which is widely used for learning neural networks, fails to leaning PUNN in general [9], [10]. The PUNN is learned using optimization algorithm such as Genetic Algorithm (GA), Particle Swarm Optimization (PSO) and LeapFrog Optimization (LFOP) [10].…”

Section: ) Dynamic Objective Functionmentioning

confidence: 99%

See 2 more Smart Citations

Inference of S-system models of genetic networks using Product Unit Neural Networks

Murata

Koshino

Mitamura

et al. 2008

2008 IEEE International Conference on Systems, Man and Cybernetics

View full text Add to dashboard Cite

Abstract-In this study, we proposed the method of inference of genetic networks which expresses the regulation of genes. The proposed method does not solve the differential equations, learns the genetic networks using Product-Unit-Neural-Network (PUNN) and infer the S-system model of genetic networks which describes a set of differential equations. The experimental results show the proposal method is 160 times faster than the previous method which estimated S-system model of genetic networks while maintaining equivalent performance to the previous method.

show abstract

Section: The Inference Methods Using Punn Modelmentioning

confidence: 99%

Section: The Inference Methods Using Punn Modelmentioning

confidence: 99%

See 1 more Smart Citation

Inference of S-system models of genetic networks using Product Unit Neural Networks

Murata

Koshino

Mitamura

et al. 2008

2008 IEEE International Conference on Systems, Man and Cybernetics

View full text Add to dashboard Cite

show abstract

“…Hence, they are advantageous in comparison to an EA-based algorithm that needs to simulated mutation and crossover operators for real-valued weight vector [110]. It was found that PSO guides a population of the FNN weight vectors towards an optimum population [161,162]. Hence, many researchers resorted to working on swarm based metaheuristics for the FNN optimization.…”

Section: Weight Optimizationmentioning

confidence: 99%

Metaheuristic design of feedforward neural networks: A review of two decades of research

Ojha

Abraham

Snel

2017

Engineering Applications of Artificial Intelligence

441

154

View full text Add to dashboard Cite

Over the past two decades, the feedforward neural network (FNN) optimization has been a key interest among the researchers and practitioners of multiple disciplines. The FNN optimization is often viewed from the various perspectives: the optimization of weights, network architecture, activation nodes, learning parameters, learning environment, etc. Researchers adopted such different viewpoints mainly to improve the FNN's generalization ability. The gradient-descent algorithm such as backpropagation has been widely applied to optimize the FNNs. Its success is evident from the FNN's application to numerous real-world problems. However, due to the limitations of the gradient-based optimization methods, the metaheuristic algorithms including the evolutionary algorithms, swarm intelligence, etc., are still being widely explored by the researchers aiming to obtain generalized FNN for a given problem. This article attempts to summarize a broad spectrum of FNN optimization methodologies including conventional and metaheuristic approaches. This article also tries to connect various research directions emerged out of the FNN optimization practices, such as evolving neural network (NN), cooperative coevolution NN, complex-valued NN, deep learning, extreme learning machine, quantum NN, etc. Additionally, it provides interesting research challenges for future research to cope-up with the present information processing era.

show abstract

“…The size of the final network (513 weights) that was used in this work even makes global optimization techniques like Particle Swarm Optimization or Genetic Algorithms infeasible. Consequently for a network of the presented size, higher order units such as product units cannot be incorporated due to the increased amount of local minima, requiring global optimization techniques (Ismail & Engelbrecht, 2000). But also with only sigmoid units, based on the possibility of backpropagation getting stuck in local minima, always a set of at least 10 networks with random initial parameters were trained.…”

Section: Network Trainingmentioning

confidence: 99%