Enhancing data quality to mine credible patterns

Abstract

The importance of big data is widely accepted in various fields. Organisations spend a lot of money to collect, process and mine the data to identify patterns. These patterns facilitate their future decision-making process to improve the organisational performance and profitability. However, among discovered patterns, there are some meaningless and misleading patterns which restrict the effectiveness of decision-making process. The presence of data discrepancies, noise and outliers also impacts the quality of discovered patterns and leads towards missing strategic goals and objectives. Quality inception of these discovered patterns is vital before utilising them in making predictions, decision-making process or strategic planning. Mining useful and credible patterns over social media is a challenging task. Often, people spread targeted content for character assassination or defamation of brands. Recently, some studies have evaluated the credibility of information over social media based on users’ surveys, experts’ judgement and manually annotating Twitter tweets to predict credibility. Unfortunately, due to the large volume and exponential growth of data, these surveys and annotation-based information credibility techniques are not efficiently applicable. This article presents a data quality and credibility evaluation framework to determine the quality of individual data instances. This framework provides a way to discover useful and credible patterns using credibility indicators. Moreover, a new Twitter bot detection algorithm is proposed to classify tweets generated by Twitter bots and real users. The results of conducted experiments showed that the proposed model generates a positive impact on improving classification accuracy and quality of discovered patterns.

On the Evaluation Framework of Comprehensive Trust for Data Interaction in Intermodal Transport

“…Then, the corresponding evaluation models and detailed processes are defined. Here, the assessment method is a comprehensive assessment approach, including both a qualitative assessment and a quantitative assessment [7].…”

“…In fact, the outcome of a business analysis task is directly influenced by the quality of input data, especially in complex analysis tasks; thus, it directly affects the security of data applications. For example, decision-making and predictions based on patterns and trends with poor quality or failing to capture real insights and images of current operational status, can lead to unnecessary risks and losses for transportation organizations, and may have effect on the safety and convenience of passengers' travelling experience [7]. Thirdly, the trust measurement should be dynamically updated with data flow between entities in the network environment, considering both static and dynamic evidence for different types of entities.…”

The Problem of Data Extraction in Social Media: A Theoretical Framework

“…It involves examining relationships between different users or actors, analyzing content, and analyzing engagement to understand how people interact. By understanding social media trends and behaviors, organizations can better understand their customers and create better strategies to reach them [21,22]. This section gives an overview of the different types of analyses that can be done on social media data.…”

Environmental Analysis of Hemorrhagic Fever in Iraq Using Machine Learning

“…This enables drawing more precise and reliable inferences from the data [48]. Refer to Table 1 and Table 2.…”