Efficiency Improvement of Classification Model Based on Altered K-Means Using PCA and Outlier

Jung, Se-Hoon; Kim, Jong-Chan

doi:10.1142/s0218194019400047

Cited by 6 publications

(9 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The conditions of an observed value include

that is the set of input data,

that is a mean, and that is standard deviation. When clustering proceeds with 1 or higher for K, the number of optimized clusters, it will satisfy Equation (3) [ 66 ].

□…”

Section: Proposed Reinforcement K-means Algorithmmentioning

confidence: 99%

See 1 more Smart Citation

A Novel Model on Reinforce K-Means Using Location Division Model and Outlier of Initial Value for Lowering Data Cost

Jung

Lee

2020

Entropy

Self Cite

View full text Add to dashboard Cite

Today, semi-structured and unstructured data are mainly collected and analyzed for data analysis applicable to various systems. Such data have a dense distribution of space and usually contain outliers and noise data. There have been ongoing research studies on clustering algorithms to classify such data (outliers and noise data). The K-means algorithm is one of the most investigated clustering algorithms. Researchers have pointed out a couple of problems such as processing clustering for the number of clusters, K, by an analyst through his or her random choices, producing biased results in data classification through the connection of nodes in dense data, and higher implementation costs and lower accuracy according to the selection models of the initial centroids. Most K-means researchers have pointed out the disadvantage of outliers belonging to external or other clusters instead of the concerned ones when K is big or small. Thus, the present study analyzed problems with the selection of initial centroids in the existing K-means algorithm and investigated a new K-means algorithm of selecting initial centroids. The present study proposed a method of cutting down clustering calculation costs by applying an initial center point approach based on space division and outliers so that no objects would be subordinate to the initial cluster center for dependence lower from the initial cluster center. Since data containing outliers could lead to inappropriate results when they are reflected in the choice of a center point of a cluster, the study proposed an algorithm to minimize the error rates of outliers based on an improved algorithm for space division and distance measurement. The performance experiment results of the proposed algorithm show that it lowered the execution costs by about 13–14% compared with those of previous studies when there was an increase in the volume of clustering data or the number of clusters. It also recorded a lower frequency of outliers, a lower effectiveness index, which assesses performance deterioration with outliers, and a reduction of outliers by about 60%.

show abstract

“…The conditions of an observed value include

that is the set of input data,

that is a mean, and that is standard deviation. When clustering proceeds with 1 or higher for K, the number of optimized clusters, it will satisfy Equation (3) [ 66 ].

□…”

Section: Proposed Reinforcement K-means Algorithmmentioning

confidence: 99%

“…Algorithm 1 shows the entire algorithm of the system to which the classification technique of data proposed in the study was applied [ 5 , 66 , 67 ]. Most of data for classification is basically multi-dimensional and multi-variate with individuals connected to one another.…”

Section: Proposed Reinforcement K-means Algorithmmentioning

confidence: 99%

A Novel Model on Reinforce K-Means Using Location Division Model and Outlier of Initial Value for Lowering Data Cost

Jung

Lee

2020

Entropy

Self Cite

View full text Add to dashboard Cite

show abstract

“…Starting with the third initial value, the distance from the other objects is calculated using the same method as MMK. The probability calculation is then repeated to assign initial values [53].…”

Section: K-means Algorithmmentioning

confidence: 99%

“…This study used an unsupervised learning-based altered K-means algorithm [33,53] to analyze pre-treated transmission line tower IoT sensor data. Note, however, that K-means algorithms have the disadvantage of lacking a data classification structure that has been optimized for application in Big Data analysis.…”

Section: Clustering Level : Altered K-means Algorithmmentioning

confidence: 99%

See 1 more Smart Citation

A Novel on Transmission Line Tower Big Data Analysis Model Using Altered K-means and ADQL

Jung

2019

Sustainability

Self Cite

View full text Add to dashboard Cite

This study sought to propose a big data analysis and prediction model for transmission line tower outliers to assess when something is wrong with transmission line tower big data based on deep reinforcement learning. The model enables choosing automatic cluster K values based on non-labeled sensor big data. It also allows measuring the distance of action between data inside a cluster with the Q-value representing network output in the altered transmission line tower big data clustering algorithm containing transmission line tower outliers and old Deep Q Network. Specifically, this study performed principal component analysis to categorize transmission line tower data and proposed an automatic initial central point approach through standard normal distribution. It also proposed the A-Deep Q-Learning algorithm altered from the deep Q-Learning algorithm to explore policies based on the experiences of clustered data learning. It can be used to perform transmission line tower outlier data learning based on the distance of data within a cluster. The performance evaluation results show that the proposed model recorded an approximately 2.29%~4.19% higher prediction rate and around 0.8% ~ 4.3% higher accuracy rate compared to the old transmission line tower big data analysis model.

show abstract

Privacy‐preserving clustering of unstructured big data for cloud‐based enterprise search solutions

Zobaed

Salehi

2022

Concurrency and Computation

View full text Add to dashboard Cite

Summary Cloud‐based enterprise search services (e.g., Amazon Kendra) are enchanting to big data owners by providing them with convenient search solutions over their enterprise big datasets. However, individuals and businesses dealing with confidential big data (e.g., criminal reports) are reluctant to fully embrace such cloud services due to valid data privacy concerns. Solutions based on client‐side encryption have been developed to mitigate these concerns. Nonetheless, such solutions hinder data processing, especially, data clustering, which is pivotal in applications such as real‐time search on large corpora (e.g., big datasets). To cluster encrypted big data, we propose privacy‐preserving clustering schemes, called ClusPr, for three forms of unstructured datasets, namely static, semi‐dynamic, and dynamic. ClusPr functions based on statistical characteristics of the datasets to: (A) determine the suitable number of clusters; (B) populate the clusters with topically relevant tokens; and (C) adapt the cluster set based on the dynamism of the underlying dataset. Experimental results, obtained from evaluating ClusPr against other schemes in the literature, on three different test datasets demonstrate between 30%$$ 30\% $$ and 60%$$ 60\% $$ improvement on the cluster coherency. Moreover, we notice that employing ClusPr within a privacy‐preserving enterprise search system can reduce the search time by up to 78%$$ 78\% $$, while improving the search accuracy by up to 35%$$ 35\% $$.

show abstract

Efficiency Improvement of Classification Model Based on Altered K-Means Using PCA and Outlier

Cited by 6 publications

References 30 publications

A Novel Model on Reinforce K-Means Using Location Division Model and Outlier of Initial Value for Lowering Data Cost

A Novel Model on Reinforce K-Means Using Location Division Model and Outlier of Initial Value for Lowering Data Cost

A Novel on Transmission Line Tower Big Data Analysis Model Using Altered K-means and ADQL

Privacy‐preserving clustering of unstructured big data for cloud‐based enterprise search solutions

Contact Info

Product

Resources

About