Improvement of ID3 Algorithm Based on Simplified Information Entropy and Coordination Degree

Abstract

Abstract:The decision tree algorithm is a core technology in data classification mining, and ID3 (Iterative Dichotomiser 3) algorithm is a famous one, which has achieved good results in the field of classification mining. Nevertheless, there exist some disadvantages of ID3 such as attributes biasing multi-values, high complexity, large scales, etc. In this paper, an improved ID3 algorithm is proposed that combines the simplified information entropy based on different weights with coordination degree in rough set theory. The traditional ID3 algorithm and the proposed one are fairly compared by using three common data samples as well as the decision tree classifiers. It is shown that the proposed algorithm has a better performance in the running time and tree structure, but not in accuracy than the ID3 algorithm, for the first two sample sets, which are small. For the third sample set that is large, the proposed algorithm improves the ID3 algorithm for all of the running time, tree structure and accuracy. The experimental results show that the proposed algorithm is effective and viable.

Evaluating the Impact of GINI Index and Information Gain on Classification using Decision Tree Classifier Algorithm*

“…For a dataset with one class label, will be 1 and ( ) is 0. Hence the Entropy of homogenous data set is zero [8]. If the entropy is higher the uncertainty/impurity/mixing is higher [9].…”

“…The feature with highest information gain is the best feature to be selected for split. Assuming that there are V different values for a feature f, |L v | represents the subset of L with f=v, Information gain after splitting L on a feature f is measured as [8].…”

Optimization for prediction model of palm oil land suitability using spatial decision tree algorithm

“…Pemodelan klasifikasi adalah proses ekstraksi data kesesuaian lahan yang telah ada [18], yang dalam algoritma SDT menggunakan entropi untuk menumbuhkan pohon keputusan. Berdasarkan hal tersebut, suatu variabel dalam suatu dataset yang memiliki sifat heterogenitas yang tinggi menyebakan nilai entropinya tinggi, dan sebaliknya jika data dalam suatu variabel bersifat homogen maka nilai entropinya akan rendah atau bahkan bernilai 0 [28]. Nilai entropi yang tinggi pada suatu variabel menyebabkan variabel tersebut memiliki peluang yang kecil untuk dijadikan sebagai simpul akar/ internal, seperti halnya variabel drainase dan kapasitas tukar kation.…”

Improvement of Criminal Identification by Smart Optimization Method

“…A hyperparameter is an internal parameter of a classifier method, such as the box constraint of DT or a support vector machine, or maybe the learning rate of a robust classification ensemble. The goal of these parameters can highly affect the performance of a classifier; BO uses a fit function [9]. For Model selection, depend on Hyper-parameters, features, train a classifier like a decision tree, using a dataset, this process needs to choose the best feature set and hyperparameters by applying K fold DT.…”