Sampling for Big Data Profiling: A Survey

Liu, Zhicheng; Zhang, Aoqian

doi:10.1109/access.2020.2988120

Cited by 26 publications

(12 citation statements)

References 103 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…From this, we conclude that the number of machines and their properties are not sufficient to achieve better execution time, but rather it requires more efficient methods and strategies. This is what (Liu and Zhang, 2020) confirmed in his proposed work. From results obtained by Pandey and Shukla (2019), our proposed work remains better in terms of increase in precision, as well as in execution time, although in their experiments they used small dataset.…”

Section: Experiments Results and Discussionsupporting

confidence: 84%

“…Among the works presented in this survey (Approximate cluster computing for big data analysis) are somewhat similar to our proposed work. Thanks to a survey realized by Liu and Zhang (2020) they confirmed us again that sampling methods reduce the volume of big data more effectively and help to speed up its processing. So, it plays an important role in the era of big data, now and in the future.…”

Section: The Final Version Of Our Proposed Workmentioning

confidence: 89%

“…Thus, mathematical statistics is the most important in data science and particularly in big data analytics (Bucchianico et al , 2019; Weihs and Ickstadt, 2018; HLG-BAS, 2011). The reason may be due to its function, as mathematical statistics reveal correlations between statistical groups, and its function is pivotal to “reduce size” to better understand data, and thus extract information with greater precision and more quickly; as a result, statisticians and data science experts went to the use of statistical sampling technique (Rojas et al , 2017; Liu and Zhang, 2020; Mahmud et al , 2020).…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Dynamic Distributed and Parallel Machine Learning algorithms for big data mining processing

Djafri

2021

DTA

View full text Add to dashboard Cite

PurposeThis work can be used as a building block in other settings such as GPU, Map-Reduce, Spark or any other. Also, DDPML can be deployed on other distributed systems such as P2P networks, clusters, clouds computing or other technologies.Design/methodology/approachIn the age of Big Data, all companies want to benefit from large amounts of data. These data can help them understand their internal and external environment and anticipate associated phenomena, as the data turn into knowledge that can be used for prediction later. Thus, this knowledge becomes a great asset in companies' hands. This is precisely the objective of data mining. But with the production of a large amount of data and knowledge at a faster pace, the authors are now talking about Big Data mining. For this reason, the authors’ proposed works mainly aim at solving the problem of volume, veracity, validity and velocity when classifying Big Data using distributed and parallel processing techniques. So, the problem that the authors are raising in this work is how the authors can make machine learning algorithms work in a distributed and parallel way at the same time without losing the accuracy of classification results. To solve this problem, the authors propose a system called Dynamic Distributed and Parallel Machine Learning (DDPML) algorithms. To build it, the authors divided their work into two parts. In the first, the authors propose a distributed architecture that is controlled by Map-Reduce algorithm which in turn depends on random sampling technique. So, the distributed architecture that the authors designed is specially directed to handle big data processing that operates in a coherent and efficient manner with the sampling strategy proposed in this work. This architecture also helps the authors to actually verify the classification results obtained using the representative learning base (RLB). In the second part, the authors have extracted the representative learning base by sampling at two levels using the stratified random sampling method. This sampling method is also applied to extract the shared learning base (SLB) and the partial learning base for the first level (PLBL1) and the partial learning base for the second level (PLBL2). The experimental results show the efficiency of our solution that the authors provided without significant loss of the classification results. Thus, in practical terms, the system DDPML is generally dedicated to big data mining processing, and works effectively in distributed systems with a simple structure, such as client-server networks.FindingsThe authors got very satisfactory classification results.Originality/valueDDPML system is specially designed to smoothly handle big data mining classification.

show abstract

Section: Experiments Results and Discussionsupporting

confidence: 84%

Section: The Final Version Of Our Proposed Workmentioning

confidence: 89%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Dynamic Distributed and Parallel Machine Learning algorithms for big data mining processing

Djafri

2021

DTA

View full text Add to dashboard Cite

show abstract

“…Sampling techniques other than random sampling provide view point to focus on different characteristics of data without compromising quality in data exploration. Liu and Zhang (2020) presented a survey based on sampling and profiling over big data. Authors advocated that sampling technologies play an important role in the era of big data and it is indispensable step in big data processing in future.…”

Section: A Reviewmentioning

confidence: 99%

Sampling-based estimation method for parameter estimation in big data business era

Alim

Shukla

2020

JAMR

View full text Add to dashboard Cite

PurposeThis paper aims to present sample-based estimation methodologies to compute the confidence interval for the mean size of the content of material communicated on the digital social media platform in presence of volume, velocity and variety. Confidence interval acts as a tool of machine learning and managerial decision-making for coping up big data.Design/methodology/approachRandom sample-based sampling design methodology is adapted and mean square error is computed on the data set. Confidence intervals are calculated using the simulation over multiple data sets. The smallest length confidence interval is the selection approach for the most efficient in the scenario of big data.FindingsResultants of computations herein help to forecast the future need of web-space at data-centers for anticipation, efficient management, developing a machine learning algorithm for predicting better quality of service to users. Finding supports to develop control limits as an alert system for better use of resources (memory space) at data centers. Suggested methodologies are efficient enough for future prediction in big data setup.Practical implicationsIn IT sector, the startup with the establishment of data centers is the current trend of business. Findings herein may help to develop a forecasting system and alert system for optimal decision-making in the enhancement and share of the business.Originality/valueThe contribution is an original piece of thought, idea and analysis, deriving motivation from references appended.

show abstract

“…Data mining is the process of extracting information and data from large databases or information repositories. The work became very interesting, attracted too many researchers and developers, and made good progress for several years [1]. Since the 1990s, the concept of data mining has been generally seen as a "mining" process.…”

Section: Introductionmentioning

confidence: 99%

Systematic Review: A State of Art ML Based Clustering Algorithms for Data Mining

Ali

Faheem

Waseem

et al. 2020

2020 IEEE 23rd International Multitopic Conference (INMIC)

View full text Add to dashboard Cite

Data mining is an unsupervised learning technique to extract the insights and hidden relationships among data. Data mining has more importance in data science and machine learning because through data mining all hidden information is shown to determine various aspects of the data set. Clustering is a data mining technique to group the data, on the basis of similarity measures. The objects or data points in a cluster are similar. Similarly, objects or data points in another cluster will also be similar. But when these clusters are compared, they are dissimilar to each other. Clustering is considered the most important unsupervised learning technique because it deals with finding a structure in a collection of unlabeled data. Clustering can be done by the different approaches like partitioning clustering, hierarchical clustering, density-based clustering, and grid-based clustering. These clustering approaches can be done by the numbers of algorithms, such as K-means clustering, Fuzzy C-means clustering, Hierarchical clustering, DBSCAN, OPTICS, STING, ROCK and CACTUS. This proposed paper contained reviews of the above techniques by using a more powerful programming language (Python) as a tool. Evidence attainable from this study is helpful for researchers to select an appropriate clustering approach based on their domain.

show abstract

Sampling for Big Data Profiling: A Survey

Cited by 26 publications

References 103 publications

Dynamic Distributed and Parallel Machine Learning algorithms for big data mining processing

Dynamic Distributed and Parallel Machine Learning algorithms for big data mining processing

Sampling-based estimation method for parameter estimation in big data business era

Systematic Review: A State of Art ML Based Clustering Algorithms for Data Mining

Contact Info

Product

Resources

About