Initialization of K-modes clustering using outlier detection techniques

Jiang, Feng; Liu, Guozhu; Du, Junwei; Sui, Yuefei

doi:10.1016/j.ins.2015.11.005

Cited by 100 publications

(64 citation statements)

References 36 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Step 3: Calculate the compactness (S W ) according to (1) Step 4: Calculate the separation of clusters (S BW ) according to (2) Step 5: Calculate the remoteness (S B ) according to (3) Step 6: Calculate the value of the following function taking into account (1)-(4)…”

Section: Algorithmmentioning

confidence: 99%

Anomaly Detection in Big Data based on Clustering

Alguliyev

Alıguliyev

Sukhostat

2017

Stat., optim. inf. comput.

View full text Add to dashboard Cite

Selection of the right tool for anomaly (outlier) detection in Big data is an urgent task. In this paper algorithms for data clustering and outlier detection that take into account the compactness and separation of clusters are provided. We consider the features of their use in this capacity. Numerical experiments on real data of different sizes demonstrate the effectiveness of the proposed algorithms.

show abstract

Section: Algorithmmentioning

confidence: 99%

Anomaly Detection in Big Data based on Clustering

Alguliyev

Alıguliyev

Sukhostat

2017

Stat., optim. inf. comput.

View full text Add to dashboard Cite

show abstract

“…In addition, One-Class SVM technology has been widely used in the field of outlier detection. (5) Clustering based method [30,31]. The method first divides the data set into several clusters, and the data points that do not belong to any cluster are outliers.…”

Section: Outlier Detectionmentioning

confidence: 99%

Correction of Outliers in Temperature Time Series Based on Sliding Window Prediction in Meteorological Sensor Network

Wang

2017

Information

View full text Add to dashboard Cite

Abstract:In order to detect outliers in temperature time series data for improving data quality and decision-making quality related to design and operation, we proposed an algorithm based on sliding window prediction. Firstly, the time series are segmented based on the sliding window. Then, the prediction model is established based on the history data to predict the future value. If the difference between a predicted value and a measured value is larger than the preset threshold value, the sequence point will be judged to be an outlier and then corrected. In this paper, the sliding window and parameter settings of the algorithm are discussed and the algorithm is verified on actual data. This method does not need to pre classify the abnormal points and perform fast, and can handle large scale data. The experimental results show that the proposed algorithm can not only effectively detect outliers in the time series of meteorological data but also improves the correction efficiency notoriously.

show abstract

“…However, there are still no generally accepted initialization methods for kmeans clustering. In [34], the initialization of k-means clustering from the point of view of anomaly detection was considered.…”

Section: Related Workmentioning

confidence: 99%

An Anomaly Detection Based on Optimization

Alguliyev¹,

Alıguliyev²,

İmamverdiyev³

et al. 2017

IJISA

View full text Add to dashboard Cite

Abstract-At present, an anomaly detection is one of the important problems in many fields. The rapid growth of data volumes requires the availability of a tool for data processing and analysis of a wide variety of data types. The methods for anomaly detection are designed to detect object's deviations from normal behavior. However, it is difficult to select one tool for all types of anomalies due to the increasing computational complexity and the nature of the data. In this paper, an improved optimization approach for a previously known number of clusters, where a weight is assigned to each data point, is proposed. The aim of this article is to show that weighting of each data point improves the clustering solution. The experimental results on three datasets show that the proposed algorithm detects anomalies more accurately. It was compared to the k-means algorithm. The quality of the clustering result was estimated using clustering evaluation metrics. This research shows that the proposed method works better than k-means on the Australia (credit card applications) dataset according to the Purity, Mirkin and F-measure metrics, and on the heart diseases dataset according to F-measure and variation of information metric.

show abstract

Initialization of K-modes clustering using outlier detection techniques

Cited by 100 publications

References 36 publications

Anomaly Detection in Big Data based on Clustering

Anomaly Detection in Big Data based on Clustering

Correction of Outliers in Temperature Time Series Based on Sliding Window Prediction in Meteorological Sensor Network

An Anomaly Detection Based on Optimization

Contact Info

Product

Resources

About