A Survey on Big Data Pre-processing

Zhi-bin, Guan; Ji, Te; Xu, Qian; Ma, Yan; Hong, Xianpei

doi:10.1109/acit-csii-bcd.2017.49

Cited by 17 publications

(8 citation statements)

References 61 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…On the other hand, the data series provided by government database can be incomplete or erroneous because of unreliable transmission. Data preparation in these contexts can be generally classified into three main groups, namely data cleansing, integration, and reduction as investigated in [111]. In FADE, cleansing is envisaged to deal with incompleteness and erroneous conditions.…”

Section: Data Preparationmentioning

confidence: 99%

See 1 more Smart Citation

Mobile computing and communications-driven fog-assisted disaster evacuation techniques for context-aware guidance support: A survey

Kurniawan

Asyhari

et al. 2021

Computer Communications

View full text Add to dashboard Cite

Section: Data Preparationmentioning

confidence: 99%

“…The data will then be shared before combined or summarized. The types of textual data collected are not necessarily identical because this may also include data integration [111]. However, preprocessing applied to the collected data requires computation resources if performed individually and locally.…”

Section: Peer-offloading Data Preparationmentioning

confidence: 99%

Mobile computing and communications-driven fog-assisted disaster evacuation techniques for context-aware guidance support: A survey

Kurniawan

Asyhari

et al. 2021

Computer Communications

View full text Add to dashboard Cite

“…Data was cleaned using tokenization, with stemming and lemmatization [12]. Noise such as hashtags, punctuation, special characters, reluctant words, along with other irrelevant data, were removed.…”

Section: Data Pre-processingmentioning

confidence: 99%

Semantic Knowledge Discovery for User Profiling for Location-Based Recommender Systems

Tao¹,

Sharma²,

Delaney³

et al. 2021

HCIS

View full text Add to dashboard Cite

This paper introduces a purposed Location-based Recommender System (LBRS) that combines sentiment analysis and topic modelling techniques to improve user profiling for enhancing recommendations of Points of Interest (POIs). Using additional feature extraction, we built user profiles from a Foursquare dataset to evaluate our model and provide recommendations based on user opinions toward venues. Our combined model performed favourably against the baseline models, with an overall improved accuracy of 0.67. The limitations were the use of one dataset and that user profiles were constructed using predicted emotions extracted as features from review data with topic modelling, rather than literal user emotions. Nevertheless, this provides a step forward in user profile and emotion scoring, contributing further to the development of LBRS in the Tourism domain.

show abstract

“…Data preparation or also known as data pre-processing is a vital process in majority of machine learning projects and activities [1]. Depending on the nature of data, there are many methods of data pre-processing that can be applied and data discretization is one of those methods [2].…”

Section: Introductionmentioning

confidence: 99%

The Impact of Fuzzy Discretization’s Output on Classification Accuracy of Random Forest Classifier

Fikri¹

2020

IJATCSE

View full text Add to dashboard Cite

Random Forest is known as among the widely used classification algorithms by researchers and machine learning enthusiast in solving classification problems. Recently, fuzzy discretization has been paired with Random Forest (RF) classifier to enhance the classification accuracy of Random Forest classifier when dealing with continuous variables. However, there are many different opinions on whether there is a need to perform discretization in data pre-processing for tree-based classifiers such as J48, Decision Tree and Random Forest. On top of that, it is known that different classification algorithms produce different classification accuracies depending on the type of data used. In other words, the output of data discretization process. Thus, to unravel this mentioned hypothesis, this study intends to shed some lights on the impact of different fuzzy discretization's output on the classification accuracy of Random Forest classifier. In this study, three version of simulations were done with different fuzzy discretization output. Those fuzzy discretization's outputs are 1) without fuzzy discretization 2) with fully fuzzy discretization and 3) with partial fuzzy discretization. Then, classification phase is done through Random Forest classifier and the classification accuracy for all the simulation versions were observed, recorded, and analyzed.

show abstract

A Survey on Big Data Pre-processing

Cited by 17 publications

References 61 publications

Mobile computing and communications-driven fog-assisted disaster evacuation techniques for context-aware guidance support: A survey

Mobile computing and communications-driven fog-assisted disaster evacuation techniques for context-aware guidance support: A survey

Semantic Knowledge Discovery for User Profiling for Location-Based Recommender Systems

The Impact of Fuzzy Discretization’s Output on Classification Accuracy of Random Forest Classifier

Contact Info

Product

Resources

About