A Novel Hadoop Security Model for Addressing Malicious Collusive Workers

Abstract

With the daily increase of data production and collection, Hadoop is a platform for processing big data on a distributed system. A master node globally manages running jobs, whereas worker nodes process partitions of the data locally. Hadoop uses MapReduce as an effective computing model. However, Hadoop experiences a high level of security vulnerability over hybrid and public clouds. Specially, several workers can fake results without actually processing their portions of the data. Several redundancy-based approaches have been proposed to counteract this risk. A replication mechanism is used to duplicate all or some of the tasks over multiple workers (nodes). A drawback of such approaches is that they generate a high overhead over the cluster. Additionally, malicious workers can behave well for a long period of time and attack later. This paper presents a novel model to enhance the security of the cloud environment against untrusted workers. A new component called malicious workers’ trap (MWT) is developed to run on the master node to detect malicious (noncollusive and collusive) workers as they convert and attack the system. An implementation to test the proposed model and to analyze the performance of the system shows that the proposed model can accurately detect malicious workers with minor processing overhead compared to vanilla MapReduce and Verifiable MapReduce (V-MR) model [1]. In addition, MWT maintains a balance between the security and usability of the Hadoop cluster.

Next-Generation Intrusion Detection and Prevention System Performance in Distributed Big Data Network Security Architectures

“…However, like most application or service modules within big data frameworks, MapReduce highlights the vast attack vectors that exist in distributed big data systems. MapReduce examples in literature include side channel attacks [19], job composition attacks [20], and malicious worker compromises in the form of distributed denial-of-service (DDoS) or replay attacks [21], Eaves dropping and data tampering [22]. Encryption is a primary countermeasure to secure transmissions and prevent data leaks between big data servers [19].…”

“…The authors note that worst-case attackers would gain as much information from a data leak as deterministic or order-preserving encryption over time [23]. While methods such as encryption and authentication help with cross-node data leaks, they do not prevent other attacks, such as DDoS and passive network eavesdropping [21]. A subsequent countermeasure is the effective design and implementation of intrusion detection and prevention systems [14].…”