Traditional machine learning and big data analytics in virtual screening: a comparative study

Hussin, Sahar K.; Omar, Yasser M. K.; Abdel-Mageid, Salah; Marie, Mahmoud

doi:10.19101/ijacr.2019.940150

Cited by 11 publications

(3 citation statements)

References 54 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Spark can compare the cost of different methods and automatically select the optimal data recovery method. Method and the equivariant method are relatively stable, and the AST method has a certain degree of improvement in F1 value compared with the equivariant method [19].…”

Section: Experimental Analysis Of the Feature Large Datasetmentioning

confidence: 99%

Clustering Algorithm for Big Datasets with Mixed Attribute Features under Spark

Wang¹

2022

Mathematical Problems in Engineering

View full text Add to dashboard Cite

This paper presents an in-depth study and analysis of large datasets of mixed and attribute features under Spark using a large dataset clustering algorithm. The classical algorithm K-means based on division and the density-based clustering algorithm DPC, which has become more popular in recent years, are selected as the research objects of this paper. Secondly, the original K-means algorithm is improved by combining holdout validation and K-means++ method to address the shortcomings of the K-means algorithm that the number of class clusters K needs to be set in advance and the initial class cluster centers are chosen randomly, which leads to unstable iterations and slow convergence of clustering results. The similarity matrix will be continuously updated during the iterative process. It mainly refers to the process of dividing objects into multiple classes according to the degree of similarity between objects. After the division, the objects within the class are like each other, while the objects between the classes are different from each other. The comparison experiments of the improved algorithm before and after the MovieLens dataset are conducted to verify that the new algorithm has better performance in terms of clustering accuracy and efficiency. Again, to address the drawback that the clustering results in the DPC algorithm rely heavily on the subjective selection of the truncation distance parameter cd, and it is difficult to handle datasets with complex distribution and large density variation, the algorithm can generate the optimal cd adaptively by combining K-nearest neighbors and introducing the distance comparison quantity, which has a better performance by considering the overall and local distribution of the data. The feasibility of the improved method is verified by validating the algorithm with artificial datasets and UCI datasets as well as separation tests. Finally, the parallelized design and implementation of the improved K-means algorithm and CDPC-KNN algorithm are completed by building a Spark clustering environment, and the parallelized algorithm is verified to have much better data processing capability and be more adaptable to the clustering analysis of large-scale data by comparing algorithm string parallelism experiments.

show abstract

Section: Experimental Analysis Of the Feature Large Datasetmentioning

confidence: 99%

Clustering Algorithm for Big Datasets with Mixed Attribute Features under Spark

Wang¹

2022

Mathematical Problems in Engineering

View full text Add to dashboard Cite

show abstract

“…K-mean clustering is one of the simplest nonsupervised learning algorithms, which was first proposed by Macqueen in 1967. It has been applied by many researchers to solve some of the problems of known groups [28]. is technique classifies a particular dataset into a certain number of groups.…”

Section: Related Workmentioning

confidence: 99%

Handling Imbalance Classification Virtual Screening Big Data Using Machine Learning Algorithms

Hussin¹,

Abdel-Mageid

Alkhalil

et al. 2021

Complexity

Self Cite

View full text Add to dashboard Cite

Virtual screening is the most critical process in drug discovery, and it relies on machine learning to facilitate the screening process. It enables the discovery of molecules that bind to a specific protein to form a drug. Despite its benefits, virtual screening generates enormous data and suffers from drawbacks such as high dimensions and imbalance. This paper tackles data imbalance and aims to improve virtual screening accuracy, especially for a minority dataset. For a dataset identified without considering the data’s imbalanced nature, most classification methods tend to have high predictive accuracy for the majority category. However, the accuracy was significantly poor for the minority category. The paper proposes a K-mean algorithm coupled with Synthetic Minority Oversampling Technique (SMOTE) to overcome the problem of imbalanced datasets. The proposed algorithm is named as KSMOTE. Using KSMOTE, minority data can be identified at high accuracy and can be detected at high precision. A large set of experiments were implemented on Apache Spark using numeric PaDEL and fingerprint descriptors. The proposed solution was compared to both no-sampling method and SMOTE on the same datasets. Experimental results showed that the proposed solution outperformed other methods.

show abstract

“…Clustering, as a subdomain of unsupervised machine learning, entails the grouping of data points into clusters or categories based on shared characteristics, thereby allowing for the exploration of latent patterns and the identification of homogeneous subsets within the data [8][9][10]. This fundamental process has found applications across diverse domains, including but not limited to marketing, healthcare, finance, and scientific research [11,12]. Its importance has been further accentuated with the advent of big data, where traditional data processing techniques prove inadequate in the face of sheer volume, velocity, and variety [13][14][15][16][17].…”

Section: Introductionmentioning

confidence: 99%

Advancements in big data clustering: methods, applications, and insights

2023

TIS

View full text Add to dashboard Cite

The digital age has given rise to an unprecedented influx of data, marking the era of big data. In this landscape, clustering has emerged as a critical element of data analysis, enabling the discovery of latent patterns in vast datasets. This review paper explores the state-of-the-art in big data clustering, encompassing influential research, methodologies, advantages, and limitations. The paper highlights the significant advantages brought by different clustering algorithms, spanning domains from smart grids and education to e-commerce and different operations. However, it also acknowledges limitations such as scalability issues and generalization challenges, underlining the importance of addressing these constraints for future research.

show abstract

Traditional machine learning and big data analytics in virtual screening: a comparative study

Cited by 11 publications

References 54 publications

Clustering Algorithm for Big Datasets with Mixed Attribute Features under Spark

Clustering Algorithm for Big Datasets with Mixed Attribute Features under Spark

Handling Imbalance Classification Virtual Screening Big Data Using Machine Learning Algorithms

Advancements in big data clustering: methods, applications, and insights

Contact Info

Product

Resources

About