A Pearson’s correlation coefficient based decision tree and its parallel implementation

Mu, Yashuang; Liu, Xiaodong; Wang, Lidong

doi:10.1016/j.ins.2017.12.059

Cited by 239 publications

(99 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…27,31 The cluster may contain thousands of computers and each computer offers a local storage and computation. 2,26 The greatest strength of MapReduce is decomposing the task on a big data into many small, single, and inherent tasks in many machine nodes of a cluster. 2,26 The greatest strength of MapReduce is decomposing the task on a big data into many small, single, and inherent tasks in many machine nodes of a cluster.…”

Section: The Mechanics Of Mapreducementioning

confidence: 99%

“…Take the open-source software Hadoop 29 as an example. 26,34 Users can flexibly implement these functions when they apply a parallel computing strategy to their own applications. 33 Each Map phase takes a small file divided from the original file as its inputs; meanwhile, each Map phase contains a Map function.…”

Section: The Mechanics Of Mapreducementioning

confidence: 99%

“…However, when these methods are directly applied to deal with large-scale data sets, the competitiveness is lost due to some limiting factors such as memory capacity, execution time, and data complexity. 26 To be able to meet the challenges, the first thing is to address the limiting factors that happened in traditional splitting rules. [20][21][22][23][24][25] Therefore, it is unavoidable to learn a classifier based on fuzzy decision trees for dealing with large-scale data sets.…”

mentioning

confidence: 99%

“…2 Fuzzy decision trees have highly transparent and user-friendly characteristics 19 ; meanwhile, the amount of data produced in our daily life is showing a fleetness increasing tendency. 26 One effective parallel computing program applied in many fields of data analysis is MapReduce. 26 To be able to meet the challenges, the first thing is to address the limiting factors that happened in traditional splitting rules.…”

mentioning

confidence: 99%

“…[20][21][22][23][24][25] Therefore, it is unavoidable to learn a classifier based on fuzzy decision trees for dealing with large-scale data sets. 26 To be able to meet the challenges, the first thing is to address the limiting factors that happened in traditional splitting rules. A parallel computing or distributed computing technique becomes a common and trustworthy scheme.…”

mentioning

confidence: 99%

See 4 more Smart Citations

A parallel tree node splitting criterion for fuzzy decision trees

Liu

Wang

et al. 2019

Concurrency and Computation

Self Cite

View full text Add to dashboard Cite

Fuzzy decision trees are one of the most important extensions of decision trees for symbolic knowledge acquisition by fuzzy representation. Many fuzzy decision trees employ fuzzy information gain as a measure to construct the tree node splitting criteria. These criteria play a critical role in the construction of decision trees. However, many of the criteria can only work well on small-scale or medium-scale data sets, and cannot directly deal with large-scale data sets on the account of some limiting factors such as memory capacity, execution time, and data complexity.Parallel computing is one way to overcome these problems; in particular, MapReduce is one mainstream solution of parallel computing. In this paper, we design a parallel tree node splitting criterion (MR-NSC) based on fuzzy information gain via MapReduce, which is completed equivalent to the traditional unparallel splitting rule. The experimental studies verify the equivalency between the proposed MR-NSC algorithm and the traditional unparallel way through 22 UCI benchmark data sets. Furthermore, the feasibility and parallelism are also studied on two large-scale data sets. KEYWORDSfuzzy decision trees, fuzzy information gain, MapReduce, parallel computing INTRODUCTIONDecision trees are one of the most well-known researches to describe a decision-making process in light of existing knowledge. Each branch of a decision tree can be transformed into a decision rule, and all these decision rules can generate a decision rule base. The popularity of decision trees mainly arises from that the decision rules are more readily comprehensible than some other decision-making models such as neural networks. 1,2 Based on decision trees, there are many extensions 3 : Most of them are extensions or improvements of the well-known ID3 algorithm 4 and CART algorithm. 5 Fuzzy decision trees are one of the most popular extensions. They combine the symbolic decision trees with approximate reasoning offered by fuzzy representation. 6 The intent is to exploit the complementary advantages of the comprehensibility of decision trees and the uncertain information of fuzzy representation. 7,8Based on some splitting mechanisms, fuzzy decision trees recursively partition the training data into several subsets with some similar or same outputs in a top-down way. In particular, as one of the splitting mechanisms, fuzzy information theory makes a widespread influence in the growth of fuzzy decision trees. Weber 9 presented a well-known fuzzy Iterative Dichotomiser 3 (ID3) algorithm by modifying the information gain measure used to split a tree node for fuzzy representation. Umanol et al 10 proposed a new algorithm based on the probability of membership values to generate a fuzzy decision tree from numerical data, in which the fuzzy sets for each attribute are predefined by users. Ichihashi et al 11 realized fuzzy partitions by extracting fuzzy reasoning rules. An algebraic method to facilitate incremental learning is also employed. As knowledge inferences must be newly defined in fuz...

show abstract

Section: The Mechanics Of Mapreducementioning

confidence: 99%

Section: The Mechanics Of Mapreducementioning

confidence: 99%

mentioning

confidence: 99%

mentioning

confidence: 99%

mentioning

confidence: 99%

See 3 more Smart Citations

A parallel tree node splitting criterion for fuzzy decision trees

Liu

Wang

et al. 2019

Concurrency and Computation

Self Cite

View full text Add to dashboard Cite

show abstract

Data‐Driven–Based Internal Temperature Estimation for Lithium‐Ion Battery Under Variant State‐of‐Charge via Electrochemical Impedance Spectroscopy

et al. 2022

View full text Add to dashboard Cite

Internal temperature estimation is critical to the safe operation of lithium‐ion batteries (LIBs), and electrochemical impedance spectroscopy (EIS)‐based methods have been demonstrated to be promising. However, accurate internal temperature estimation under variant state‐of‐charge (SoC) is still challenging due to the combined impact of temperature and SoC on the EIS. Accordingly, this work proposes a novel EIS‐based internal temperature estimation approach, for which SoC‐insensitive EIS features are quantitatively selected and utilized for temperature estimation using support vector regression (SVR) with unknown SoC. First, the EIS feature selection is performed to select SoC‐insensitive features from the imaginary of impedance spectrum. Subsequently, an SVR‐based framework and a well‐trained SVR model are created to estimate the internal temperature of LIBs. The performance of the proposed model is validated by its lowest estimation error (0.57 °C) under known and unknown SoCs compared to that of the existing methodologies. The results confirmed that the proposed method holds the advantage in estimating the internal battery temperature with different SOCs.

show abstract

Research on optimization and application of Spark decision tree algorithm under cloud‐edge collaboration

Wang

Jia

Cao

2022

Int J of Intelligent Sys

View full text Add to dashboard Cite

This paper optimizes and parallelizes the C4.5 algorithm on the Spark platform under cloud-edge collaboration. First, the information entropy calculation in the definition of boundary points and Fayyad's theorem is improved by the Gini index, which reduces the number of information entropy calculations for selecting segmentation points in the traditional C4.5 algorithm continuous attribute discretization operation and simplifies the calculation formula, thus reducing the execution time of the algorithm. On this basis, the CFS algorithm is introduced to optimize the calculation of the information gain ratio, to facilitate the selection of better attributes for decision tree partitioning. The improved C4.5 algorithm is then parallelized in Spark platform. In this paper, we choose the "intelligent door guard" of cloud-edge collaboration under the epidemic prevention and control management context as the application scenario for experimental verification and make timely risk assessment for those who want to enter the place and give them a response whether they are allowed to enter or not.Experiments show that the improved parallel C4.5 algorithm reduces the running time and raises the accuracy of the algorithm, compared with the traditional C4.5 algorithm and previous improvements to it.In the application scenario of cloud-edge collaboration,

show abstract

A Pearson’s correlation coefficient based decision tree and its parallel implementation

Cited by 239 publications

References 27 publications

A parallel tree node splitting criterion for fuzzy decision trees

A parallel tree node splitting criterion for fuzzy decision trees

Data‐Driven–Based Internal Temperature Estimation for Lithium‐Ion Battery Under Variant State‐of‐Charge via Electrochemical Impedance Spectroscopy

Research on optimization and application of Spark decision tree algorithm under cloud‐edge collaboration

Contact Info

Product

Resources

About