A fast clustering algorithm based on pruning unnecessary distance computations in DBSCAN for high-dimensional data

Chen, Yewang; Tang, Shengyu; Bouguila, Nizar; Wang, Cheng; Du, Ji-Xiang; Li, Hailin

doi:10.1016/j.patcog.2018.05.030

Cited by 135 publications

(63 citation statements)

References 54 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…where w k,j , a uniformly distributed random data in [0, 1], is the weight coefficient of v f k,j , w k,j ≥ 0, and ∑ 4 k=1 w k,j = 1 . To satisfy this constraint, the conversion operations of w k,j in Equation 24are expressed as Equation 23, thus the candidates in Equation 22are rewritten as Equation (24).…”

Section: The Combined Forecasting Results By Optimized Weightsmentioning

confidence: 99%

See 1 more Smart Citation

A Compound Wind Power Forecasting Strategy Based on Clustering, Two-Stage Decomposition, Parameter Optimization, and Optimal Combination of Multiple Machine Learning Approaches

Sun

2019

Energies

View full text Add to dashboard Cite

Given the large-scale exploitation and utilization of wind power, the problems caused by the high stochastic and random characteristics of wind speed make researchers develop more reliable and precise wind power forecasting (WPF) models. To obtain better predicting accuracy, this study proposes a novel compound WPF strategy by optimal integration of four base forecasting engines. In the forecasting process, density-based spatial clustering of applications with noise (DBSCAN) is firstly employed to identify meaningful information and discard the abnormal wind power data. To eliminate the adverse influence of the missing data on the forecasting accuracy, Lagrange interpolation method is developed to get the corrected values of the missing points. Then, the two-stage decomposition (TSD) method including ensemble empirical mode decomposition (EEMD) and wavelet transform (WT) is utilized to preprocess the wind power data. In the decomposition process, the empirical wind power data are disassembled into different intrinsic mode functions (IMFs) and one residual (Res) by EEMD, and the highest frequent time series IMF1 is further broken into different components by WT. After determination of the input matrix by a partial autocorrelation function (PACF) and normalization into [0, 1], these decomposed components are used as the input variables of all the base forecasting engines, including least square support vector machine (LSSVM), wavelet neural networks (WNN), extreme learning machine (ELM) and autoregressive integrated moving average (ARIMA), to make the multistep WPF. To avoid local optima and improve the forecasting performance, the parameters in LSSVM, ELM, and WNN are tuned by backtracking search algorithm (BSA). On this basis, BSA algorithm is also employed to optimize the weighted coefficients of the individual forecasting results that produced by the four base forecasting engines to generate an ensemble of the forecasts. In the end, case studies for a certain wind farm in China are carried out to assess the proposed forecasting strategy.

show abstract

Section: The Combined Forecasting Results By Optimized Weightsmentioning

confidence: 99%

“…DBSCAN, a density-based spatial data clustering method, is widely used in image processing [22] and high-dimensional data procession [23,24]. The aims of the DBSCAN method are to group the candidate points into core points, border points, or outliers.…”

Section: Density Based Spatial Clustering Of Applications With Noise mentioning

confidence: 99%

A Compound Wind Power Forecasting Strategy Based on Clustering, Two-Stage Decomposition, Parameter Optimization, and Optimal Combination of Multiple Machine Learning Approaches

Sun

2019

Energies

View full text Add to dashboard Cite

show abstract

“…In the future, we will focus on how to combine the different features more effectively and select a better classification model for the combined feature. Moreover, we will improve the proposed method by data preparation techniques, such as clustering [51][52][53], normalization, and data cleaning.…”

Section: Discussionmentioning

confidence: 99%

Steganalysis of Adaptive Multi-Rate Speech Based on Extreme Gradient Boosting

et al. 2020

Self Cite

View full text Add to dashboard Cite

Steganalysis of adaptive multi-rate (AMR) speech is a hot topic for controlling cybercrimes grounded in steganography in related speech streams. In this paper, we first present a novel AMR steganalysis model, which utilizes extreme gradient boosting (XGBoost) as the classifier, instead of support vector machines (SVM) adopted in the previous schemes. Compared with the SVM-based model, this new model can facilitate the excavation of potential information from the high-dimensional features and can avoid overfitting. Moreover, to further strengthen the preceding features based on the statistical characteristics of pulse pairs, we present the convergence feature based on the Markov chain to reflect the global characterization of pulse pairs, which is essentially the final state of the Markov transition matrix. Combining the convergence feature with the preceding features, we propose an XGBoost-based steganalysis scheme for AMR speech streams. Finally, we conducted a series of experiments to assess our presented scheme and compared it with previous schemes. The experimental results demonstrate that the proposed scheme is feasible, and can provide better performance in terms of detecting the existing steganography methods based on AMR speech streams.

show abstract

“…The photons are checked until the signal photon clusters are completely expanded or there are no photons left to check. A photon is defined as a "noise photon" if it does not belong to any classified signal cluster [21,39,40]. The clustering result and fitness value are calculated using the values of eps and MinPts, and then optimized by an iterative process.…”

Section: Methodsmentioning

confidence: 99%

Particle Swarm Optimization-Based Noise Filtering Algorithm for Photon Cloud Data in Forest Area

et al. 2019

View full text Add to dashboard Cite

The Ice, Cloud, and land Elevation Satellite-2 (ICESat-2), which is equipped with the Advanced Topographic Laser Altimeter System (ATLAS), was launched successfully in 15 September 2018. The ATLAS represents a micro-pulse photon-counting laser system, which is expected to provide more comprehensive and scientific data for carbon storage. However, the ATLAS system is sensitive to the background noise, which poses a tremendous challenge to the photon cloud noise filtering. Moreover, the Density Based Spatial Clustering of Applications with Noise (DBSCAN) is a commonly used algorithm for noise removal from the photon cloud but there has not been an in-depth study on its parameter selection yet. This paper presents an automatic photon cloud filtering algorithm based on the Particle Swarm Optimization (PSO) algorithm, which can be used to optimize the two key parameters of the DBSCAN algorithm instead of using the manual parameter adjustment. The Particle Swarm Optimization Density Based Spatial Clustering of Applications with Noise (PSODBSCAN) algorithm was tested at different laser intensities and laser pointing types using the MATLAS dataset of the forests located in Virginia, East Coast, and the West Coast, USA. The results showed that the PSODBSCAN algorithm and the localized statistical algorithm were effective in identifying the background noise and preserving the signal photons in the raw MATLAS data. Namely, the PSODBSCAN achieved the mean F value of 0.9759, and the localized statistical algorithm achieved the mean F value of 0.6978. For both laser pointing types and laser intensities, the proposed algorithm achieved better results than the localized statistical algorithm. Therefore, the PSODBSCAN algorithm could support the MATLAS photon cloud data noise filtering applicably without manually selecting parameters.

show abstract

A fast clustering algorithm based on pruning unnecessary distance computations in DBSCAN for high-dimensional data

Cited by 135 publications

References 54 publications

A Compound Wind Power Forecasting Strategy Based on Clustering, Two-Stage Decomposition, Parameter Optimization, and Optimal Combination of Multiple Machine Learning Approaches

A Compound Wind Power Forecasting Strategy Based on Clustering, Two-Stage Decomposition, Parameter Optimization, and Optimal Combination of Multiple Machine Learning Approaches

Steganalysis of Adaptive Multi-Rate Speech Based on Extreme Gradient Boosting

Particle Swarm Optimization-Based Noise Filtering Algorithm for Photon Cloud Data in Forest Area

Contact Info

Product

Resources

About