Determination of the Optimal Neural Network Transfer Function for Response Surface Methodology and Robust Design

Le, Tuan-Ho; Jang, Hyeonae; Shin, Sangmun

doi:10.3390/app11156768

Cited by 6 publications

(4 citation statements)

References 47 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The choice of the transfer function can significantly impact the efficiency and accuracy of the model that is translating each input to a specific output. Sigmoid function is one of the common transfer functions used in neural networks, which maps the input to a value between 0 and 1 [79,80]. The sigmoid function is commonly used in classification tasks, producing outputs that can be interpreted as probabilities.…”

Section: Transfer Function Impactsmentioning

confidence: 99%

Critical Review of Neural Network Generations and Models Design

Yousif,

Yousif

2023

Preprint

View full text Add to dashboard Cite

In recent years, Neural networks are increasingly deployed in various fields to learn complex patterns and make accurate predictions. However, designing an effective neural network model is a challenging task that requires careful consideration of various factors, including architecture, optimization method, and regularization technique. This paper aims to comprehensively overview the state-of-the-art artificial neural network (ANN) generation and highlight key challenges and opportunities in machine learning applications. It provides a critical analysis of current neural network model design methodologies, focusing on the strengths and weaknesses of different approaches. Also, it explores the use of different learning approaches, including convolutional neural networks (CNN), deep neural networks (DNN), and recurrent neural networks (RNN) in image recognition, natural language processing, and time series analysis. Besides, it discusses the benefits of choosing the ideal values for the different components of ANN, such as the number of Input/output layers, hidden layers number, activation function type, epochs number, and model type selection, which help improve the model performance and generalization. Furthermore, it identifies some common pitfalls and limitations of existing design methodologies, such as overfitting, lack of interpretability, and computational complexity. Finally, it proposes some directions for future research, such as developing more efficient and interpretable neural network architectures, improving the scalability of training algorithms, and exploring the potential of new paradigms, such as Spiking Neural Networks, quantum neural networks, and neuromorphic computing.

show abstract

Section: Transfer Function Impactsmentioning

confidence: 99%

Critical Review of Neural Network Generations and Models Design

Yousif,

Yousif

2023

Preprint

View full text Add to dashboard Cite

show abstract

“…Selecting an RSM design is important to get better prediction and optimum results as there are many different designs in RSM. On top of that, selection of architecture in the neural network especially the number of hidden layers or nodes could affect the performance of the process [19,20]. Nevertheless, none of the reported studies have compared the performance of the adsorption process with different RSM designs and ANN training methods.…”

Section: Introductionmentioning

confidence: 99%

Statistical Modeling and Optimization of Process Parameters for 2,4-Dichlorophenoxyacetic Acid Removal by Using AC/PDMAEMA Hydrogel Adsorbent: Comparison of Different RSM Designs and ANN Training Methods

Dahlan

Azhar

Hassan

et al. 2022

Water

View full text Add to dashboard Cite

In this study, the response surface methodology (RSM) and artificial neural network (ANN) were employed to study the adsorption process of 2,4-dichlorophenoxyacetic acid (2,4-D) by using modified hydrogel, i.e., activated carbon poly(dimethylaminoethyl methacrylate) (AC/PDMAEMA hydrogel). The effect of pH, the initial concentration of 2,4-D and the activated carbon content on the removal of 2,4-D and adsorption capacity were investigated through the face-centered composite design (FCCD), optimal design and two-level factorial design. The response surface plot suggested that higher removal of 2,4-D and adsorption capacity could be achieved at the higher initial concentration of 2,4-D and lower pH and activated carbon content. The modeling and optimization for the adsorption process of 2,4-D were also carried out by different design methods of RSM and different training methods of ANN. It was found that among the three design methods of RSM, the optimal design has the highest accuracy for the prediction of 2,4-D removal and adsorption capacity (R2 = 0.9958 and R2 = 0.9998, respectively). The numerical optimization of the optimal design found that the maximum removal of 2,4-D and adsorption capacity of 65.01% and 65.29 mg/g, respectively, were obtained at a pH of 3, initial concentration of 2,4-D of 94.52 mg/L and 2.5 wt% of activated carbon. Apart from the optimization of process parameters, the neural network architecture was also optimized by trial and error with different numbers of hidden neurons in the layers to obtain the best performance of the response. The optimization of the neural network was performed with different training methods. It was found that among the three training methods of the ANN model, the Bayesian Regularization method had the highest R2 and lowest mean square error (MSE) with the optimum network architecture of 3:9:2. The optimum condition obtained from RSM was also simulated with the optimized neural network architecture to validate the responses and adequacy of the RSM model.

show abstract

“…Recently, Arungpadang et al [44] proposed a hybrid neural network-genetic algorithm to predict process parameters. Le et al proposed NN-based response function estimation (NRFE) identifies a new screening procedure to obtain the best transfer function in an NN structure using a desirability function family while determining its associated weight parameters [45]. Le and Shin propose an NN-based estimation method as a RD modeling approach.…”

Section: Introductionmentioning

confidence: 99%

Robust Process Parameter Design Methodology: A New Estimation Approach by Using Feed-Forward Neural Network Structures and Machine Learning Algorithms

Dai

Jang

et al. 2022

Applied Sciences

Self Cite

View full text Add to dashboard Cite

In robust design (RD) modeling, the response surface methodology (RSM) based on the least-squares method (LSM) is a useful statistical tool for estimating functional relationships between input factors and their associated output responses. Neural network (NN)-based models provide an alternative means of executing input-output functions without the assumptions necessary with LSM-based RSM. However, current NN-based estimation methods do not always provide suitable response functions. Thus, there is room for improvement in the realm of RD modeling. In this study, a new NN-based RD modeling procedure is proposed to obtain the process mean and standard deviation response functions. Second, RD modeling methods based on the feed-forward back-propagation neural network (FFNN), cascade-forward back-propagation neural network (CFNN), and radial basis function network (RBFN) are proposed. Third, two simulation studies are conducted using a given true function to verify the proposed three methods. Fourth, a case study is examined to illustrate the potential of the proposed approach. In conclusion, a comparative analysis of the three feed-forward NN structure-based modeling methods and conventional LSM-based RSM proposed in this study showed that the proposed methods were significantly lower in the expected quality loss (EQL) and various variability indicators.

show abstract

Determination of the Optimal Neural Network Transfer Function for Response Surface Methodology and Robust Design

Cited by 6 publications

References 47 publications

Critical Review of Neural Network Generations and Models Design

Critical Review of Neural Network Generations and Models Design

Statistical Modeling and Optimization of Process Parameters for 2,4-Dichlorophenoxyacetic Acid Removal by Using AC/PDMAEMA Hydrogel Adsorbent: Comparison of Different RSM Designs and ANN Training Methods

Robust Process Parameter Design Methodology: A New Estimation Approach by Using Feed-Forward Neural Network Structures and Machine Learning Algorithms

Contact Info

Product

Resources

About