Data Preprocessing: A preliminary step for web data mining

Jamshed, Huma; Khan, M. Sadiq Ali; Khurram, Muhammad; Inayatullah, Syed; Athar, Sameen

doi:10.17993/3ctecno.2019.specialissue2.206-221

Cited by 19 publications

(13 citation statements)

References 6 publications

(6 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Data preparation is one of the most critical phases in any methodological approach based on data mining. During this phase, the quality of the data is improved by cleaning, integrating, scaling, reducing dimensionality, transforming, and selecting relevant features from raw data, enhancing machine learning algorithms’ performance [ 25 , 26 ]. Data preparation can be seen as a phase that transforms real-world raw inconsistent and incomplete data into an understandable format.…”

Section: Materials and Methodsmentioning

confidence: 99%

Multi-Class Classification of Medical Data Based on Neural Network Pruning and Information-Entropy Measures

Sánchez-Gutiérrez

González-Pérez

2022

Entropy

View full text Add to dashboard Cite

Medical data includes clinical trials and clinical data such as patient-generated health data, laboratory results, medical imaging, and different signals coming from continuous health monitoring. Some commonly used data analysis techniques are text mining, big data analytics, and data mining. These techniques can be used for classification, clustering, and machine learning tasks. Machine learning could be described as an automatic learning process derived from concepts and knowledge without deliberate system coding. However, finding a suitable machine learning architecture for a specific task is still an open problem. In this work, we propose a machine learning model for the multi-class classification of medical data. This model is comprised of two components—a restricted Boltzmann machine and a classifier system. It uses a discriminant pruning method to select the most salient neurons in the hidden layer of the neural network, which implicitly leads to a selection of features for the input patterns that feed the classifier system. This study aims to investigate whether information-entropy measures may provide evidence for guiding discriminative pruning in a neural network for medical data processing, particularly cancer research, by using three cancer databases: Breast Cancer, Cervical Cancer, and Primary Tumour. Our proposal aimed to investigate the post-training neuronal pruning methodology using dissimilarity measures inspired by the information-entropy theory; the results obtained after pruning the neural network were favourable. Specifically, for the Breast Cancer dataset, the reported results indicate a 10.68% error rate, while our error rates range from 10% to 15%; for the Cervical Cancer dataset, the reported best error rate is 31%, while our proposal error rates are in the range of 4% to 6%; lastly, for the Primary Tumour dataset, the reported error rate is 20.35%, and our best error rate is 31%.

show abstract

Section: Materials and Methodsmentioning

confidence: 99%

Multi-Class Classification of Medical Data Based on Neural Network Pruning and Information-Entropy Measures

Sánchez-Gutiérrez

González-Pérez

2022

Entropy

View full text Add to dashboard Cite

show abstract

“…The data preprocessing is the most critical step in analytics as almost 70% of the analysis time is consumed in preparation and preprocessing of the data [56,57]. This stage has been further divided into the following data preprocessing steps.…”

Section: Data Quality Evaluationmentioning

confidence: 99%

Enhancing data quality to mine credible patterns

Imran

Awada

2021

Journal of Information Science

View full text Add to dashboard Cite

The importance of big data is widely accepted in various fields. Organisations spend a lot of money to collect, process and mine the data to identify patterns. These patterns facilitate their future decision-making process to improve the organisational performance and profitability. However, among discovered patterns, there are some meaningless and misleading patterns which restrict the effectiveness of decision-making process. The presence of data discrepancies, noise and outliers also impacts the quality of discovered patterns and leads towards missing strategic goals and objectives. Quality inception of these discovered patterns is vital before utilising them in making predictions, decision-making process or strategic planning. Mining useful and credible patterns over social media is a challenging task. Often, people spread targeted content for character assassination or defamation of brands. Recently, some studies have evaluated the credibility of information over social media based on users’ surveys, experts’ judgement and manually annotating Twitter tweets to predict credibility. Unfortunately, due to the large volume and exponential growth of data, these surveys and annotation-based information credibility techniques are not efficiently applicable. This article presents a data quality and credibility evaluation framework to determine the quality of individual data instances. This framework provides a way to discover useful and credible patterns using credibility indicators. Moreover, a new Twitter bot detection algorithm is proposed to classify tweets generated by Twitter bots and real users. The results of conducted experiments showed that the proposed model generates a positive impact on improving classification accuracy and quality of discovered patterns.

show abstract

“…Once transformed, data is key to more agile and assertive decisions, a remarkable feature that denotes the ability to analyse and implement real-time changes required by the Fourth Industrial Revolution [8] [12]. In this context, this new scenario requires higher quality and reliability of the information, where data is evaluated according to integrity, consistency, credibility, accuracy and clarity in its parameters, so that any analysis can discover models and patterns, relevant characteristics or trends in organisational historical records, the "Log of events" [13].…”

Section: Industry 40 and Data Gathering And Analysismentioning

confidence: 99%

A Multi-Criteria Approach for FMEA in Product Development in Industry 4.0

Andrade

Leite

Canciglieri

et al. 2020

Advances in Transdisciplinary Engineering

View full text Add to dashboard Cite

In Industry 4.0, current trends on data analysis and real-time processing are intertwined with the use of development tools while conceiving products. The quality of information plays a pivotal role in this process, as the heterogeneity of unprocessed data may result in issues that increase the product’s cost and development time. In this context, the Failure Mode & Effect Analysis (FMEA) is a tool that aids decision-making by presenting pre-processed useful data that can improve product design, maintainability and manufacturing. FMEA criteria, although standardised, has ramifications that have different impacts and weight in an organisation, which might end up resulting in subjective evaluations. To cope with this issue and improve the quality of the information in product development, this research proposes a Multi-Criteria approach to define the importance of FMEA criteria and their impacts in organisations in a real industrial scenario. This research is applied using a case on a Brazilian electronics manufacturer, using AHP and TOPSIS Multi-Criteria Decision Making (MCDM) methods. Findings show that by traditional methods different evaluators end up generating different data and weights on FMEA, resulting in different results. In this sense, the application of the Multi-Criteria methods ends up ranking the importance of criteria and evaluating the inputs from all departments, returning more precise information. Furthermore, the weighting scale of FMEA has been shifted to a customised scale for the organisation in the study, based on AHP, more suitable to their needs and following their perception, to support the decision-makers most assertively.

show abstract

Data Preprocessing: A preliminary step for web data mining

Cited by 19 publications

References 6 publications

Multi-Class Classification of Medical Data Based on Neural Network Pruning and Information-Entropy Measures

Multi-Class Classification of Medical Data Based on Neural Network Pruning and Information-Entropy Measures

Enhancing data quality to mine credible patterns

A Multi-Criteria Approach for FMEA in Product Development in Industry 4.0

Contact Info

Product

Resources

About