Quality and Security in Big Data: Challenges as opportunities to build a powerful wrap-up solution

Abstract

Quality and Security are two major issues in Big Data that pose many challenges. High volume, heterogeneity and high speed of data generation and processing are, amongst others, common challenges that must be addressed before setting up any data quality management system or data security system. This document provides an overview of data quality and data security in a Big Data context and highlights the conflicts that may exist during the implementation of these systems. Such a conflict makes the setting up of such systems even more complex and the reflection into new solutions becomes a major prerequisite. In this paper, we consider these challenges to present a global solution to evaluate the quality of data without impacting data security and without it becoming a barrier.

Untitled

“…Recently, Otoum et al [21] exploited RL techniques on a hybrid IDS framework in wireless networks [56]. Considering a big data-driven [65] IDS approach, the authors compare and demonstrate the better performances of the RL-based IDS compared to the previously existing adaptive ML-based ones. In the field of autonomous vehicles [62], Xing et al exploit a trust evaluation model to support a two-level IDS.…”

Towards a Powerful Solution for Data Accuracy Assessment in the Big Data Context

“…As we presented in [32], to create a sample of a dataset, different techniques exist such as Simple Random Sampling, Stratified Sampling, Cluster Sampling, Multistage Sampling, Systematic Sampling, etc. Several techniques can be used together to create an effective sample, the main rules are that the sample must be representative of all data and all data units must have the same chance of being selected in the sample.…”