Dynamic Nonparametric Random Forest Using Covariance

Abstract

As the representative ensemble machine learning method, the Random Forest (RF) algorithm has widely been used in diverse applications on behalf of the fast learning speed and the high classification accuracy. Research on RF can be classified into two categories: (1) improving the classification accuracy and (2) decreasing the number of trees in a forest. However, most of papers related to the performance improvement of RF have focused on improving the classification accuracy. Only some papers have focused on reducing the number of trees in a forest. In this paper, we propose a new Covariance-Based Dynamic RF algorithm, called C-DRF. Compared to the previous works, while ensuring the good-enough classification accuracy, the proposed C-DRF algorithm reduces the number of trees. Specifically, by computing the covariance between the number of trees in a forest and F-measure at each iteration, the proposed algorithm determines whether to increase the number of trees composing a forest. To evaluate the performance of the proposed C-DRF algorithm, we compared the learning time, the test time, and the memory usage with the original RF algorithm under the different areas of datasets. Under the same or higher classification accuracy, it is shown that the proposed C-DRF algorithm improves the performance of the original RF algorithm by as much as 58.68% at learning time, 47.91% at test time, and 68.06% in memory usage on average. As a practical application area, we also show that the proposed C-DRF algorithm is more efficient than the state-of-the-art RF algorithms in Network Intrusion Detection (NID) area.

Data Driven Approach for Eye Disease Classification with Machine Learning

“…Tree based methods such as Decision Tree and Random Forest are non-parametric classifiers and, therefore, can be applied to the data whose distribution in not know. As in the medical domain collecting normally distributed data is almost impossible, non-parametric methods are always helpful [54]. However, some other classifiers such as Neural Network, Naïve Bayes and Support Vector Machines have also been very popular in classifying diagnostic data but use assumptions to simplify the learning process and sometimes lead to a higher error rate.…”

Research on a Decision Tree Classification Algorithm Based on Granular Matrices

“…In order to further optimize the classification performance of decision trees, many optimized decision tree classification models have been proposed through the integration of commonly used attribute selection operators with other theories. Examples include a decision tree integrating information gain and the Gini index [18], a decision tree integrating information entropy and the correlation coefficient [19] or the covariance [20], a decision tree based on ant colony optimization [21], and a decision tree with parallelization [22]. These optimized decision tree models extended the decision tree classification method and improved decision tree theory.…”

Optimizing Random Forests: Spark Implementations of Random Genetic Forests

“…Since RF works well with high-dimensional data [10], this robust ensemble ML algorithm is widely used with medical data [3], gene expression data [20], image processing, pattern recognition, document retrieval [22], and many other fields that inherently have high-dimensional data. More recently, the RF algorithm has also been widely used with the classification of network intrusion data [4,8].…”