Neural network method for failure detection with skewed class distribution

Carvajal, Karina; Chacón, Máx; Mery, Domingo; Acuña, Gonzalo

doi:10.1784/insi.46.7.399.55578

Cited by 20 publications

(8 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Consequently, the classification error decreases for the prevalent class and at the same time increases for the infrequent classes. One of the potential solutions to this problem involves undersampling the prevalent class by eliminating highly correlated cases [67]. In the case of the Naive Bayes classifier, the imbalanced training dataset may lead to an overfitting of the learned parameters [68] and to poor classification performance.…”

Section: Imbalanced Distribution Of Classesmentioning

confidence: 99%

Comparison of the Novel Probabilistic Self-Optimizing Vectorized Earth Observation Retrieval Classifier with Common Machine Learning Algorithms

Musiał

Bojanowski

2022

Remote Sensing

View full text Add to dashboard Cite

The Vectorized Earth Observation Retrieval (VEOR) algorithm is a novel algorithm suited to the efficient supervised classification of large Earth Observation (EO) datasets. VEOR addresses shortcomings in well-established machine learning methods with an emphasis on numerical performance. Its characteristics include (1) derivation of classification probability; (2) objective selection of classification features that maximize Cohen’s kappa coefficient (κ) derived from iterative “leave-one-out” cross-validation; (3) reduced sensitivity of the classification results to imbalanced classes; (4) smoothing of the classification probability field to reduce noise/mislabeling; (5) numerically efficient retrieval based on a pre-computed look-up vector (LUV); and (6) separate parametrization of the algorithm for each discrete feature class (e.g., land cover). Within this study, the performance of the VEOR classifier was compared to other commonly used machine learning algorithms: K-nearest neighbors, support vector machines, Gaussian process, decision trees, random forest, artificial neural networks, AdaBoost, Naive Bayes and Quadratic Discriminant Analysis. Firstly, the comparison was performed using synthetic 2D (two-dimensional) datasets featuring different sample sizes, levels of noise (i.e., mislabeling) and class imbalance. Secondly, the same experiments were repeated for 7D datasets consisting of informative, redundant and insignificant features. Ultimately, the benchmarking of the classifiers involved cloud discrimination using MODIS satellite spectral measurements and a reference cloud mask derived from combined CALIOP lidar and CPR radar data. The results revealed that the proposed VEOR algorithm accurately discriminated cloud cover using MODIS data and accurately classified large synthetic datasets with low or moderate levels of noise and class imbalance. On the contrary, VEOR did not feature good classification skills for significantly distorted or for small datasets. Nevertheless, the comparisons performed proved that VEOR was within the 3–4 most accurate classifiers and that it can be applied to large Earth Observation datasets.

show abstract

Section: Imbalanced Distribution Of Classesmentioning

confidence: 99%

Comparison of the Novel Probabilistic Self-Optimizing Vectorized Earth Observation Retrieval Classifier with Common Machine Learning Algorithms

Musiał

Bojanowski

2022

Remote Sensing

View full text Add to dashboard Cite

show abstract

“…After this reduction there still remained a large number of coefficients for each stage, but it was seen that they contained redundant information. With this large number of coefficients and only 56 cases, it was not possible to carry out a principal component analysis (26), and thus the characteristics were reduced by discarding correlation coefficients that had a percentage of correlation higher than a threshold value (27). This percentage is called percentage of similarity between the coefficients.…”

Section: Preprocessing Stepmentioning

confidence: 99%

Detection of patients with functional dyspepsia using wavelet transform applied to their electrogastrogram

Chacón

Curilem

Acuña³

et al. 2009

Braz J Med Biol Res

View full text Add to dashboard Cite

The aim of the present study was to develop a classifier able to discriminate between healthy controls and dyspeptic patients by analysis of their electrogastrograms. Fifty-six electrogastrograms were analyzed, corresponding to 42 dyspeptic patients and 14 healthy controls. The original signals were subsampled, filtered and divided into the pre-, post-, and prandial stages. A time-frequency transformation based on wavelets was used to extract the signal characteristics, and a special selection procedure based on correlation was used to reduce their number. The analysis was carried out by evaluating different neural network structures to classify the wavelet coefficients into two groups (healthy subjects and dyspeptic patients). The optimization process of the classifier led to a linear model. A dimension reduction that resulted in only 25% of uncorrelated electrogastrogram characteristics gave 24 inputs for the classifier. The prandial stage gave the most significant results. Under these conditions, the classifier achieved 78.6% sensitivity, 92.9% specificity, and an error of 17.9 +/- 6% (with a 95% confidence level). These data show that it is possible to establish significant differences between patients and normal controls when time-frequency characteristics are extracted from an electrogastrogram, with an adequate component reduction, outperforming the results obtained with classical Fourier analysis. These findings can contribute to increasing our understanding of the pathophysiological mechanisms involved in functional dyspepsia and perhaps to improving the pharmacological treatment of functional dyspeptic patients.

show abstract

Section: Preprocessing Stepmentioning

confidence: 99%

Detection of patients with functional dyspepsia using wavelet transform applied to their electrogastrogram

Chacón¹,

Curilem²,

Acuña³

et al. 2009

Braz J Med Biol Res

View full text Add to dashboard Cite

Neural network method for failure detection with skewed class distribution

Cited by 20 publications

References 7 publications

Comparison of the Novel Probabilistic Self-Optimizing Vectorized Earth Observation Retrieval Classifier with Common Machine Learning Algorithms

Comparison of the Novel Probabilistic Self-Optimizing Vectorized Earth Observation Retrieval Classifier with Common Machine Learning Algorithms

Detection of patients with functional dyspepsia using wavelet transform applied to their electrogastrogram

Detection of patients with functional dyspepsia using wavelet transform applied to their electrogastrogram

Contact Info

Product

Resources

About