In-Memory Data Anonymization Using Scalable and High Performance RDD Design

Abstract

Recent studies in data anonymization techniques have primarily focused on MapReduce. However, these existing MapReduce based approaches often suffer from many performance overheads due to their inappropriate use of data allocation, expensive disk I/O access and network transfer, and no support for iterative tasks. We propose “SparkDA” which is a new novel anonymization technique that is designed to take the full advantage of Spark platform to generate privacy-preserving anonymized dataset in the most efficient way possible. Our proposal offers a better partition control, in-memory operation and cache management for iterative operations that are heavily utilised for data anonymization processing. Our proposal is based on Spark’s Resilient Distributed Dataset (RDD) with two critical operations of RDD, such as FlatMapRDD and ReduceByKeyRDD, respectively. The experimental results demonstrate that our proposal outperforms the existing approaches in terms of performance and scalability while maintaining high data privacy and utility levels. This illustrates that our proposal is capable to be used in a wider big data applications that demands privacy.

Scalable, High-Performance, and Generalized Subtree Data Anonymization Approach for Apache Spark

“…Figure 12 presents the KLD values of our proposed subtree generalization implementation on Adult and Irish datasets. The KLD values increase with the increase of k-group size and are very close to the comparative approaches discussed [25,49].…”

“…We observed that the overall trends for the DM to the DM values observed in other similar approaches in [25,51]. As the k-group size increases, more records are part of an EC, and thus records are less distinguishable from each other.…”

“…Though the I E score is highest at (log 2 k) in [51], the I E score of our proposal works better than the scheme proposed by [29]. The performance of our approach is close to the higher end of standard and achieves much higher privacy levels achieved by the scheme proposed by [25]. As the entropy represents the information content of data change, the entropy after data anonymization should be higher than the entropy before the anonymization which is the phenomenon observed in our results.…”

“…To address the overheads associated with MapReduce, several Spark-based approaches have been proposed in recent years [18,[25][26][27][28]. In [29], the authors proposed the INCOG-NITO framework for full-domain generalization using Spark RDDs.…”

“…The experiments are carry out using two datasets: US Census dataset (i.e., Adult dataset) [34] and Irish Census dataset [43], larger datasets are created using the similar proposed approach proposed [25] for the experiments. Tables 5 and 6 illustrates each quasi-identifiable attribute (QID) we used in our experiments and generalization level (GL) of each QID obtained from the taxonomy trees for Adult and Irish datasets.…”