Strategies to develop robust neural network models: Prediction of flash point as a case study

Abstract

Artificial neural network (ANN) is one of the most widely used methods to develop accurate predictive models based on artificial intelligence and machine learning. In the present study, the important practical aspects of developing a reliable ANN model e.g. appropriate assignment of the number of neurons, number of hidden layers, transfer function, training algorithm, dataset division and initialization of the network are discussed. As a case study, predictability of the flash point for a dataset of 740 organic compounds using ANNs was investigated via a total number of 484220ANNs to allow covering a wide range of parameters affecting the performance of an ANN. Among all studied parameters, the number of neurons or layers was found to be the most important parameters to develop a reliable ANN with low overfitting risk. To evaluate appropriate number of neurons and layers, a value of equal or greater than 10 for the ratio of the training samples to the ANN constants was suggested as a rule of thumb. More ever, a strategy for evaluation of the authentic performance of ANNs and deciding about the reliability of an ANN model was proposed which is applicable to other models developed by supervised learning. Based on the introduced considerations, an ANN model was proposed for predicting the flash point of pure organic compounds. According to the results, the new model was found to produce the lowest error compared to other available models.

Assessment of lipid peroxidation and artificial neural network models in early Alzheimer Disease diagnosis

“…The ANN models were constructed from the 24 independent variables (gender and age as factors, 22 analytes as covariables), and 1 dependent variable (participant group (MCI-AD/healthy control)). In the first step, the dataset was randomly divided into training sample (70%), testing sample (15%) and validating sample (15%) [18], before model development. The training sample is used to train the network in several iterations improving the ANN performance.…”

“…Artificial neural network (ANN) constitutes a promising statistical tool since it is flexible and can model highly non-linear systems, in which the relationships between variables are unknown or very complex [16][17][18]. The ANN models simulate the learning process carried out by the neurons, establishing connections among different variables, and allowing a complex data analysis through mathematical functions ANN.…”

A Neural Network QSPR Model for Accurate Prediction of Flash Point of Pure Hydrocarbons

“…The relationship between the selected descriptors and the FP was studied via feed‐forward artificial neural networks with one hidden layer containing 1–5 neurons for two cases 1) using only the selected descriptors as model inputs, and 2) using NBP in addition to the selected descriptors as the inputs of the model. To avoid overfitting and develop robust and reliable neural network models, we exploited the strategies proposed by Alibakhshi . Based on that, we first assigned 75 % of the dataset for training, 13 % for validation and 12 % to test the model.…”

“…For the neural network models which in the first step resulted an AARE % of lower than 2.5 %, the initially assigned model parameters were recorded to use in the next step for evaluating the appropriateness of training and also the authentic performances of the models, based on the method proposed by Alibakhshi . Accordingly, for the initially selected models using the same initially assigned parameters, training was repeated for 20 different randomly selected training, validation and test sets and the average of 20 repeats were used as the authentic performance of those models.…”

Thermal and Thermomechanical Responses Prediction of a Steel Ladle Using a Back‐Propagation Artificial Neural Network Combining Multiple Orthogonal Arrays

“…The artificial neural network (ANN) provides a promising technique to fulfil this target, and is one of the most extensively used methods in prediction based artificial intelligence and machine learning . It can be categorized as feed forward or recurrent.…”