Random Projection Clustering on Streaming Data

Carraher, Lee A.; Wilsey, Philip A.; Moitra, Anindya; Dey, Sayantan

doi:10.1109/icdmw.2016.0105

Cited by 8 publications

(5 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This algorithm iterates the function and recalculates the centroids until the common process is obtained, such as a constant function or many iterations. The modification process is influenced by the concept of group cohesion, where elements in one group should be more similar than elements in other groups [24][25][26][27]. Changing the cluster function not only adjusts the cluster but also affects the location of the centroids.…”

Section: Resultsmentioning

confidence: 99%

Decoding characteristics of key physical properties in silver nanoparticles by attaining centroids for cytotoxicity prediction through data cleansing

Desai,

Bandopadhyaya,

Ashok

et al. 2024

Mach. Learn.: Sci. Technol.

View full text Add to dashboard Cite

This research underscores the profound impact of data cleansing, ensuring dataset integrity and providing a structured foundation for unravelling unraveling convoluted connections between diverse physical properties and cytotoxicity. As the scientific community delves deeper into this interplay, it becomes clear that precise data purification is a fundamental aspect of investigating parameters within datasets. The study presents the need for data filtration in the background of Artificial Intelligence and Machine Learning (AI/ (ML) that has widened its horizon into the field of biological application through the amalgamation of predictive systems and algorithms that delve into the intricate characteristics of cytotoxicity of nanoparticles. The reliability and accuracy of models in the ML AI/ML landscape hinge on the quality of input data, making data cleansing a critical component of the pre-processing pipeline. The main encounter faced here is the lengthy, broad and complex datasets that has have to be toned down for further studies. Through a thorough data cleansing process, this study addresses the complexities arising from diverse sources, resulting in a refined dataset. The filtration process employs K-meansK-means clustering to derive centroids, revealing the correlation between the physical properties of nanoparticles, viz, concentration, zeta potential, hydrodynamic diameter, morphology, and absorbance andwavelength, and cytotoxicity outcomes measured in terms of cell viability. The cell lines considered for determining the centroid values that predicts the cytotoxicity of silver nanoparticles are human and animal cell lines which were categorized as normal and carcinoma type. The objective of the study is to simplify the high-dimensional data for accurate analysis of the parameters that affect the cytotoxicity of silver NPs through finding out the centroids. 

show abstract

Section: Resultsmentioning

confidence: 99%

Decoding characteristics of key physical properties in silver nanoparticles by attaining centroids for cytotoxicity prediction through data cleansing

Desai,

Bandopadhyaya,

Ashok

et al. 2024

Mach. Learn.: Sci. Technol.

View full text Add to dashboard Cite

show abstract

“…In terms of applications, variants of the RP algorithm have been successfully applied to address some of the most important challenges of big data systems, including privacy protection [27,28], handling of high-dimensional data [6,29], and system scalability [7,30,31], among many others.…”

Section: Random Projection Variantsmentioning

confidence: 99%

Tuning Database-Friendly Random Projection Matrices for Improved Distance Preservation on Specific Data

López-Sánchez

Bodt

Lee³

et al. 2021

Appl Intell

View full text Add to dashboard Cite

Random Projection is one of the most popular and successful dimensionality reduction algorithms for large volumes of data. However, given its stochastic nature, different initializations of the projection matrix can lead to very different levels of performance. This paper presents a guided random search algorithm to mitigate this problem. The proposed method uses a small number of training data samples to iteratively adjust a projection matrix, improving its performance on similarly distributed data. Experimental results show that projection matrices generated with the proposed method result in a better preservation of distances between data samples. Conveniently, this is achieved while preserving the database-friendliness of the projection matrix, as it remains sparse and comprised exclusively of integers after being tuned with our algorithm. Moreover, running the proposed algorithm on a consumer-grade CPU requires only a few seconds.

show abstract

“…The idea is to learn the manifold only on a fraction of the stream until the embedding is stable and then do the embedding by a nearest neighbor approach, eventually implemented in a stream ISOMAP algorithm (Schoeneman et al 2017). Random Projection has already been applied in the fields of non-stationary data (Carraher et al 2016;Pham et al 2017). A stream clustering algorithm called streaming-RPHas, which uses RP and localitysensitivity hashing with reasonable results was published in Carraher et al (2016).…”

Section: Related Workmentioning

confidence: 99%

“…Random Projection has already been applied in the fields of non-stationary data (Carraher et al 2016;Pham et al 2017). A stream clustering algorithm called streaming-RPHas, which uses RP and localitysensitivity hashing with reasonable results was published in Carraher et al (2016). Another work uses a Hoeffding Tree Ensemble to classify streaming data, but instead of working on high dimensions, it works on a lower dimensional space, the dimensionality reduction is done by RP Pham et al (2017).…”

Section: Related Workmentioning

confidence: 99%

“…There is ongoing research on the analysis of high dimensional data, in particular in stationary environments (Lee and Verleysen 2007;Kaban 2015). Only recently dimensionality reduction in non-stationary environments has been addressed (Carraher et al 2016). The high dimensionality of the data can lead to the phenomenon, that the distance to the nearest data point is very close to the distance to the farthest data point.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Dimensionality reduction in the context of dynamic social media data streams

2021

View full text Add to dashboard Cite

In recent years social media became an important part of everyday life for many people. A big challenge of social media is, to find posts, that are interesting for the user. Many social networks like Twitter handle this problem with so-called hashtags. A user can label his own Tweet (post) with a hashtag, while other users can search for posts containing a specified hashtag. But what about finding posts which are not labeled by the creator? We provide a way of completing hashtags for unlabeled posts using classification on a novel real-world Twitter data stream. New posts will be created every second, thus this context fits perfectly for non-stationary data analysis. Our goal is to show, how labels (hashtags) of social media posts can be predicted by stream classifiers. In particular, we employ random projection (RP) as a preprocessing step in calculating streaming models. Also, we provide a novel real-world data set for streaming analysis called NSDQ with a comprehensive data description. We show that this dataset is a real challenge for state-of-the-art stream classifiers. While RP has been widely used and evaluated in stationary data analysis scenarios, non-stationary environments are not well analyzed. In this paper, we provide a use case of RP on real-world streaming data, especially on NSDQ dataset. We discuss why RP can be used in this scenario and how it can handle stream-specific situations like concept drift. We also provide experiments with RP on streaming data, using state-of-the-art stream classifiers like adaptive random forest and concept drift detectors. Additionally, we experimentally evaluate an online principal component analysis (PCA) approach in the same fashion as we do for RP. To obtain higher dimensional synthetic streams, we use random Fourier features (RFF) in an online manner which allows us, to increase the number of dimensions of low dimensional streams.

show abstract

Random Projection Clustering on Streaming Data

Cited by 8 publications

References 24 publications

Decoding characteristics of key physical properties in silver nanoparticles by attaining centroids for cytotoxicity prediction through data cleansing

Decoding characteristics of key physical properties in silver nanoparticles by attaining centroids for cytotoxicity prediction through data cleansing

Tuning Database-Friendly Random Projection Matrices for Improved Distance Preservation on Specific Data

Dimensionality reduction in the context of dynamic social media data streams

Contact Info

Product

Resources

About