Machine learning facilitated business intelligence (Part II)

Abstract

Purpose The purpose of this paper is three-fold: to review the categories explaining mainly optimization algorithms (techniques) in that needed to improve the generalization performance and learning speed of the Feedforward Neural Network (FNN); to discover the change in research trends by analyzing all six categories (i.e. gradient learning algorithms for network training, gradient free learning algorithms, optimization algorithms for learning rate, bias and variance (underfitting and overfitting) minimization algorithms, constructive topology neural networks, metaheuristic search algorithms) collectively; and recommend new research directions for researchers and facilitate users to understand algorithms real-world applications in solving complex management, engineering and health sciences problems. Design/methodology/approach The FNN has gained much attention from researchers to make a more informed decision in the last few decades. The literature survey is focused on the learning algorithms and the optimization techniques proposed in the last three decades. This paper (Part II) is an extension of Part I. For the sake of simplicity, the paper entitled “Machine learning facilitated business intelligence (Part I): Neural networks learning algorithms and applications” is referred to as Part I. To make the study consistent with Part I, the approach and survey methodology in this paper are kept similar to those in Part I. Findings Combining the work performed in Part I, the authors studied a total of 80 articles through popular keywords searching. The FNN learning algorithms and optimization techniques identified in the selected literature are classified into six categories based on their problem identification, mathematical model, technical reasoning and proposed solution. Previously, in Part I, the two categories focusing on the learning algorithms (i.e. gradient learning algorithms for network training, gradient free learning algorithms) are reviewed with their real-world applications in management, engineering, and health sciences. Therefore, in the current paper, Part II, the remaining four categories, exploring optimization techniques (i.e. optimization algorithms for learning rate, bias and variance (underfitting and overfitting) minimization algorithms, constructive topology neural networks, metaheuristic search algorithms) are studied in detail. The algorithm explanation is made enriched by discussing their technical merits, limitations, and applications in their respective categories. Finally, the authors recommend future new research directions which can contribute to strengthening the literature. Research limitations/implications The FNN contributions are rapidly increasing because of its ability to make reliably informed decisions. Like learning algorithms, reviewed in Part I, the focus is to enrich the comprehensive study by reviewing remaining categories focusing on the optimization techniques. However, future efforts may be needed to incorporate other algorithms into identified six categories or suggest new category to continuously monitor the shift in the research trends. Practical implications The authors studied the shift in research trend for three decades by collectively analyzing the learning algorithms and optimization techniques with their applications. This may help researchers to identify future research gaps to improve the generalization performance and learning speed, and user to understand the applications areas of the FNN. For instance, research contribution in FNN in the last three decades has changed from complex gradient-based algorithms to gradient free algorithms, trial and error hidden units fixed topology approach to cascade topology, hyperparameters initial guess to analytically calculation and converging algorithms at a global minimum rather than the local minimum. Originality/value The existing literature surveys include comparative study of the algorithms, identifying algorithms application areas and focusing on specific techniques in that it may not be able to identify algorithms categories, a shift in research trends over time, application area frequently analyzed, common research gaps and collective future directions. Part I and II attempts to overcome the existing literature surveys limitations by classifying articles into six categories covering a wide range of algorithm proposed to improve the FNN generalization performance and convergence rate. The classification of algorithms into six categories helps to analyze the shift in research trend which makes the classification scheme significant and innovative.

Investigation on Evaluation and Optimization of Intelligent Business Statistical Models

“…Its core technology consists of three key parts, namely: data warehouse, data mining, and online analytical processing. With the continuous maturity of BI technology, the role of BI technology in maintaining or improving its competitiveness is also recognized by more and more enterprises [14][15]. So far, many companies or institutions have successfully applied BI in other countries.…”

Dynamic multilayer growth: Parallel vs. sequential approaches

“…Constructive algorithms involve starting with a small network architecture (typically one hidden unit) and gradually adding connections, units, or layers during training to match task complexity [18][19][20]. Conversely, pruning algorithms involve starting with a larger network where connections and units are pruned (removed) during the learning process to match task complexity (for surveys, see [21][22][23][24]).…”

“…The focus of constructive algorithms has primarily been on growing the width of a single hidden layer. Examples of applications with this focus include regression problems [32,33], classification [12,13,20,[34][35][36][37][38][39][40][41][42][43][44][45][46], and image segmentation [47] to name a few (for a list of more applications, see [19]). Conversely, approaches based on cascade correlation have offered a constructive approach that focused on growing network depth instead of width.…”

Virtual Analyzers for MI and Density Based on Neural Networks Improved through an Integrated Strategy Involving a Constructive Algorithm and Definition of Initial Weights

“…The initial estimation of the weights and the definition of the number of hidden units in the different neural models are still challenges. [19,26,27] This work presents an approach able to cope with these problems in an integrated way which is applied for the building of a soft sensor in a real polymerization system, developing and validating virtual analyzers to estimate MI and density of Linear Polyethylene (LPE) produced in an industrial unit of the Braskem Company (Camaçari-BA, Brazil) through a production process based on the "Sclairtech" technology (licensor NOVA Chemicals, Canada). The soft sensors are based on SFNN with an innovative and improved approach to neural model identification.…”