Use of Classification Algorithms in Noise Detection and Elimination

Miranda, André L. L.; García, L.; Carvalho, André Carlos Ponce de Leon Ferreira; Lorena, Ana Carolina

doi:10.1007/978-3-642-02319-4_50

Cited by 70 publications

(39 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…The performance of each classifier for the training set and test set before and after cleaning data are evaluVol.14 No. 3,2010 Journal of Advanced Computational Intelligence 299 and Intelligent Informatics Table 3. Configuration of neural networks in the experiments.…”

Section: Experiments and Resultsmentioning

confidence: 99%

“…The results asserted that after removing the class noise from the training set, the classification accuracies improved significantly. Miranda et al [3] compared three techniques for noise detection and elimination in bioinformatics data sets. The three techniques are removal of noise instances, reclassifying noise instances, and a hybrid of removal and reclassifying techniques.…”

Section: P(ν) = P(µ)p(γ)mentioning

confidence: 99%

“…They concluded that the noise removal technique provided more accurate classification Vol.14 No. 3,2010 Journal of Advanced Computational Intelligence 297 and Intelligent Informatics than the other two techniques: reclassifying and the hybrid method. Verbaeten and Assche [4] applied ensemble methods to identify and remove noisy instances from the training set in classification tasks.…”

Section: P(ν) = P(µ)p(γ)mentioning

confidence: 99%

See 2 more Smart Citations

Data Cleaning for Classification Using Misclassification Analysis

Jeatrakul¹,

Wong²,

Fung

2010

JACIII

View full text Add to dashboard Cite

In most classification problems, sometimes in order to achieve better results, data cleaning is used as a preprocessing technique. The purpose of data cleaning is to remove noise, inconsistent data and errors in the training data. This should enable the use of a better and representative data set to develop a reliable classification model. In most classification models, unclean data could sometime affect the classification accuracies of a model. In this paper, we investigate the use of misclassification analysis for data cleaning. In order to demonstrate our concept, we have used Artificial Neural Network (ANN) as the core computational intelligence technique. We use four benchmark data sets obtained from the University of California Irvine (UCI) machine learning repository to investigate the results from our proposed data cleaning technique. The experimental data sets used in our experiment are binary classification problems, which are German credit data, BUPA liver disorders, Johns Hopkins Ionosphere and Pima Indians Diabetes. The results show that the proposed cleaning technique could be a good alternative to provide some confidence when constructing a classification model.

show abstract

Section: Experiments and Resultsmentioning

confidence: 99%

Section: P(ν) = P(µ)p(γ)mentioning

confidence: 99%

Section: P(ν) = P(µ)p(γ)mentioning

confidence: 99%

See 1 more Smart Citation

Data Cleaning for Classification Using Misclassification Analysis

Jeatrakul¹,

Wong²,

Fung

2010

JACIII

View full text Add to dashboard Cite

show abstract

“…In the last couple of years, the growth rate of the working data sets in domains such as bio-medical [1], space-exploration [2], geography and computer science has been in a continuously increasing process. Due to this fact, many times all this information is categorized in different classes using automated systems, leading to a very big probability of having also wrongly classified instances, also known as noises [3].…”

Section: Introductionmentioning

confidence: 99%

A Practical Approach on Cleaning-Up Large Data Sets

Barat¹,

Prelipcean²,

Gavriluţ³

2014

2014 16th International Symposium on Symbolic and Numeric Algorithms for Scientific Computing

View full text Add to dashboard Cite

In this paper we propose a noise detection system based on similarities between instances. Having a data set with instances that belongs to multiple classes, a noise instance denotes a wrongly classified record. The similarity between different labeled instances is determined computing distances between them using several metrics among the standard ones. In order to ensure that this approach is computational feasible for very large data sets, we compute distances between pairs of different labels instances that have a certain degree of similarity. This speed-up is possible through a new clustering method called BDT Clustering presented within this paper, which is based on a supervised learning algorithm.

show abstract

“…In year 2009, Miranda, Garcia and Carvalho et. al [15] focused on bioinformatics dataset. As misclassification in biological data effects the prediction performance of classifier.…”

Section: Literature Surveymentioning

confidence: 99%

Misclassification in Big Data Soft Set Environment

Arora¹,

Kaur²

2017

IJCA

View full text Add to dashboard Cite

In order to deal with classification for large data, data filtering and data cleansing are used as preprocessing methods. Generally it remove noisy data, misclassified data, errors and inconsistent data and results unreliable classification. Because sometimes cleaned data can also affect the prediction accuracy or other testing. In this paper, we performed analysis of misclassified data and identify how much data has been wrong classified. For future aspect, This misclassified data is need to be rectified to get valuable information. To demonstrate this concept, we have used Air Traffic dataset from Statistical Computing Statistical Graphics (SCSG) to examine misclassified content in data set. Five supervised classifiers are used: Support vector Machine, decision procedure, k-nearest neighbor, random forest and logistic regression. The results shows that out of these classifiers, SVM classify 86% of the data correctly and only 14% of data has misclassification.

show abstract

Use of Classification Algorithms in Noise Detection and Elimination

Cited by 70 publications

References 6 publications

Data Cleaning for Classification Using Misclassification Analysis

Data Cleaning for Classification Using Misclassification Analysis

A Practical Approach on Cleaning-Up Large Data Sets

Misclassification in Big Data Soft Set Environment

Contact Info

Product

Resources

About