Outlier Detection Based on Residual Histogram Preference for Geometric Multi-Model Fitting

Zhao, Xi; Zhang, Yun; Xie, Shoulie; Qin, Qianqing; Wu, Shiqian; Luo, Bin

doi:10.3390/s20113037

Cited by 8 publications

(5 citation statements)

References 55 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Most of the time, the key to improving the fitting accuracy is the outliers, whose residuals to all the true models in the data set are bigger than the inliers, which makes the consensus of the outliers. When the proportion of good hypotheses is high enough after the sampling process, the quantized residual preferences [44,45] of the outliers will tend to have big values, which makes the outliers gather away from the inliers in quantized residual preference space. However previously outlier detection by quantized residual preferences conducted only in preferences space, thus making the results sensitive to the sampling process.…”

Section: Preprintsmentioning

confidence: 99%

Outlier Detection by Energy Minimization in Quantized Residual Preference Space for Geometric Model Fitting

Yun,

Bin,

et al. 2024

Preprint

View full text Add to dashboard Cite

Outliers seriously affect the accuracy in geometric model fitting. Previous works in coping with outliers involve threshold selection and scale estimation. However most scale estimators suppose the inlier distribution as a Gaussian model, which usually poorly meets the cases in geometric model fitting. Outliers, considered as the points with big residuals to all the true models, share common items with big values in quantized residual preferences, thus making the outliers gather away from the inliers in quantized residual preference space. In this paper we make use of this outlier consensus in quantized residual preference space by extending the usage of energy minimization to combine the model error and the spatial smoothness in quantized residual preference space for outlier detection. The energy minimization based outlier detection process follows an alternate sampling and labeling framework. After the outlier detection, an ordinary energy minimization method is employed to optimize the inlier labels, which also follows the framework of alternate sampling and labeling. The experimental results in this study show that the energy minimization based outlier detection method can detect most of the outliers in the data. Furthermore, the proposed energy minimization based inlier segmentation process segments the inliers into different models quite accurately. Overall, the performance of the proposed method is better than most of the state-of-the-art methods.

show abstract

Section: Preprintsmentioning

confidence: 99%

Outlier Detection by Energy Minimization in Quantized Residual Preference Space for Geometric Model Fitting

Yun,

Bin,

et al. 2024

Preprint

View full text Add to dashboard Cite

show abstract

“…The algorithm works well on univariate data [132]. A similar anomaly detection approach can be used there, even though histograms for multidimensional data are computationally intensive and need a large number of operations [133], [134].…”

Section: Histogram-based Outlier Detectionmentioning

confidence: 99%

PAACDA: Comprehensive Data Corruption Detection Algorithm

et al. 2023

View full text Add to dashboard Cite

With the advent of technology, data and its analysis are no longer just values and attributes strewn across spreadsheets, they are now seen as a stepping stone to bring about revolution in any significant field. Data corruption can be brought about by a variety of unethical and illegal sources, making it crucial to develop a method that is highly effective to identify and appropriately highlight the various corrupted data existing in the dataset. Detection of corrupted data, as well as recovering data from a corrupted dataset, is a challenging problem. This requires utmost importance and if not addressed at earlier stages may pose problems in later stages of data processing with machine or deep learning algorithms. In the following work we begin by introducing the PAACDA: Proximity based Adamic Adar Corruption Detection Algorithm and consolidating the results whilst particularly accentuating the detection of corrupted data rather than outliers. Current state of the art models, such as Isolation forest, DBSCAN also called "Density-Based Spatial Clustering of Applications with Noise" and others, are reliant on fine-tuning parameters to provide high accuracy and recall, but they also have a significant level of uncertainty when factoring the corrupted data. In the present work, the authors look into the most niche performance issues of several unsupervised learning algorithms for linear and clustered corrupted datasets. Also, a novel PAACDA algorithm is proposed which outperforms other unsupervised learning benchmarks on 15 popular baselines including K-means clustering, Isolation forest and LOF (Local Outlier Factor) with an accuracy of 96.35% for clustered data and 99.04% for linear data. This article also conducts a thorough exploration of the relevant literature from the previously stated perspectives. In this research work, we pinpoint all the shortcomings of the present techniques and draw direction for future work in this field.

show abstract

“…Extremely different from other observations, the outliers often cause anomalies (Aggarwal and Yu, 2005). Outliers may affect the accuracy of the final model (Domingues et al, 2018;Zhao et al, 2020). Consequently, before feature selection and models establishment, outliers in the data should be eliminated.…”

Section: Treatment Of Outliersmentioning

confidence: 99%

Poplar’s Waterlogging Resistance Modeling and Evaluating: Exploring and Perfecting the Feasibility of Machine Learning Methods in Plant Science

et al. 2022

View full text Add to dashboard Cite

Floods, as one of the most common disasters in the natural environment, have caused huge losses to human life and property. Predicting the flood resistance of poplar can effectively help researchers select seedlings scientifically and resist floods precisely. Using machine learning algorithms, models of poplar’s waterlogging tolerance were established and evaluated. First of all, the evaluation indexes of poplar’s waterlogging tolerance were analyzed and determined. Then, significance testing, correlation analysis, and three feature selection algorithms (Hierarchical clustering, Lasso, and Stepwise regression) were used to screen photosynthesis, chlorophyll fluorescence, and environmental parameters. Based on this, four machine learning methods, BP neural network regression (BPR), extreme learning machine regression (ELMR), support vector regression (SVR), and random forest regression (RFR) were used to predict the flood resistance of poplar. The results show that random forest regression (RFR) and support vector regression (SVR) have high precision. On the test set, the coefficient of determination (R2) is 0.8351 and 0.6864, the root mean square error (RMSE) is 0.2016 and 0.2780, and the mean absolute error (MAE) is 0.1782 and 0.2031, respectively. Therefore, random forest regression (RFR) and support vector regression (SVR) can be given priority to predict poplar flood resistance.

show abstract

Outlier Detection Based on Residual Histogram Preference for Geometric Multi-Model Fitting

Cited by 8 publications

References 55 publications

Outlier Detection by Energy Minimization in Quantized Residual Preference Space for Geometric Model Fitting

Outlier Detection by Energy Minimization in Quantized Residual Preference Space for Geometric Model Fitting

PAACDA: Comprehensive Data Corruption Detection Algorithm

Poplar’s Waterlogging Resistance Modeling and Evaluating: Exploring and Perfecting the Feasibility of Machine Learning Methods in Plant Science

Contact Info

Product

Resources

About