Temperature Cycling on Simulated Annealing for Neural Network Learning

Assessment of artificial neural network and genetic programming as predictive tools

“…Therefore, to be successful, the optimization process requires a starting point obtained from a global search. A robust training process needs both the initialization and optimization processes [38]. A schematic of the hybrid ANN algorithm is presented in Fig.…”

“…Although ANNs typically build ''black box'' models, explicit formulas can be derived for a trained ANN model. A derivative-free optimization algorithm should be added to the training process of the ANN algorithm to avoid local minima, which lead to false convergence of the ANN model [38]. Some researchers have already combined ANN and global optimization algorithms to improve ANN efficiency (e.g., [41,46].…”

Practical Considerations for Simulated Annealing Implementation

“…On temperature cycling the temperature goes down and up cyclically refining the quality of the solution at each cycle. As it was indicated in (Ledesma et al, 2007), temperature cycling is beneficial for training of auto associative neural networks. Additionally, it has been pointed out (Reed & Marks, 1999) that a temperature reduction schedule inversely proportional to the logarithm of time will guarantee converge (in probability) to a global minimum, however, in practice this schedule takes too long, and it is often more efficient to repeat the algorithm a number of times using a faster schedule.…”

“…For artificial neural network training the implementation of simulated annealing requires a good random number generator, see (Press et al, 2002) to see code to implement such type of generators. The authors have suggested simulated annealing using temperature cycling for neural network training (Ledesma et al, 2007). The free software Neural Lab is a powerful tool to simulate artificial neural networks, and it can be downloaded from http:/ / www.fimee.ugto.mx/ profesores/ sledesma/ .…”

Simulated Annealing Evolution

“…As the temperature decreases, each ANN has the chance to improve its skills. If the ANNs are required to incorporate new skills, temperature cycling can be used, see [6]. Specifically, an ANN may learn by a combination of SA and some sort of hands-on experience.…”