Parallel computing for artificial neural network training

Abstract

As an enormous computing power is required to get knowledge from a large volume of data, the parallel and distributed computing is highly recommended to process them. Artificial Neural Networks (ANNs) need as much as possible data to have high accuracy, whereas parallel processing can help us to save time in ANNs training. In this paper, exemplary parallelization of artificial neural network training by dint of Java and its native socket libraries has been implemented. During the experiments, it has been noticed that Java native socket implementation tends to have memory issues when a large amount of training datasets are involved in training. It has been remarked that exemplary parallelization of artificial neural network training cannot outperform drastically when additional nodes are introduced into the system after a certain point. This is comprehensively due to the network communication complexity in the system.Corresponding Author: Name Md. Haidar Sharif Affiliation International Balkan University Address Skopje, Republic of Macedonia Phone +38923214831 Email msharif@ibu.edu.mk IntroductionParallel and distributed computing are required to get knowledge from a large volume of data. Artificial Neural Networks (ANNs) need to have high accuracy, whereas parallel processing can help us to save time in ANNs training. In the literature, many articles on parallelization of neural network pieces of training can be found. Some examples are given herewith. A parallelized Back-Propagation Neural Network (BPNN) based on MapReduce computing model was introduced by Liu et al. [1]. A cascading model based classification approach was implemented to introduce fault tolerance, data replication, and load balancing. Nodes were trained for a specific class available in training dataset. As a result, a much stronger classifier could be built by the weak classifiers in the cluster. A parallelizing technique for the training of neural networks on the cluster computer was presented by Dahl et al. [2]. A full neural network was copied at each cluster node for the eight-bit parity problem. The subset of the training data in each node was randomly selected at each epoch, which is the date and time related to a computer's clock [3]. An improvement up to a factor of 11 in terms of training time as compared to sequential training was demonstrated through the obtained results. The type of neural network problem was mentioned. A different kind of exemplary parallelization was implemented. The communication costs of the approach were analysed. But it was not recommended the node parallelism in cluster computer. A comparative study along with an own technique for the performance of exemplary parallel and node parallel strategies on a cluster computer were performed by Pethick et al. [4]. The performance of various sizes of neural networks, miscellaneous dataset sizes, and the number of processors was studied. The affirmative and the negative sides of their proposed method were discussed. The cost equation introduced in their method ...

Modelling tax consciousness evaluation in the context of economic development uncertainty

“…The highest influence of the behavioral component on the level of tax consciousness is identified for small taxpayers while it is the lowest for controlling bodies. The value of the cognitive component [2], [5], [10], [35] for all subjects of taxation is high due to the high accessibility of education, spread of lifelong learning ideas, development of advisory functions of controlling bodies, all of which had a general positive effect on the level of tax awareness. The emotional and volitional component is defined by trust in government and taxation system, and subjective nature of managerial decisions made.…”

“…The paper [16], [17] substantiates the model of taxation subjects' tax consciousness on the principles of reflexive interaction and integral estimation, which results in a strategy of raising tax consciousness level. Tax consciousness evaluation tools are represented by the methods of sociological research (questionnaires) whose results are statistically estimated [10], [15], [18], [24], analyzed to identify cause and effect relationships and regularities using the correlation and regression analysis [4], [20], [22], Evolutionary Game Theory [19]. Fuzzy simulation scenarios [14] were developed and analyzed establishing paramount importance of forming positive attitude towards taxation in order to raise the level of tax consciousness.…”

“…Cognitive component [2], [5], [8], [10], [12] Emotional & volitional component [1], [2], [3], [4], [11], [36] Behavioral component [6], [19] The first level of the hierarchy represents the objectives of evaluating the components of tax consciousness: cognitive K, emotional-volitional E, and behavioral P ones. The second level includes factors determining the previous level components, namely: tax awareness, emotions that arise in the process of taxation, attitude to taxation, guidelines on how to act in terms of taxation.…”

Parameter identification of PMSM using EKF with temperature variation tracking in automotive applications

“…To obtain more accurate parameters estimation for PMSM, many different methods have been used in literature, such as neural network, genetic algorithm, Luenberger obsever and EKF which are effective and efficient alternatives for tracking parameter variation , it may be also used for real-time estimation of parameters and state of PMSM [4]- [6].…”

Intelligent vision-based navigation system for mobile robot: A technological review

“…The nature of operations for many industrial and service applications are changing due to the advancement of sciences and technologies. These include the applicability of various state-of-the-art methods [1][2][3][4] in contributing towards the advancement. The latest feasibility of Internet of Things [5] that propels Industrial Revolution 4.0 also has contributed towards establishing better applications.…”