TrustMR: Computation integrity assurance system for MapReduce

Ulusoy, Huseyin; Kantarcıoğlu, Murat; Pattuk, Erman

doi:10.1109/bigdata.2015.7363785

Cited by 17 publications

(9 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Therefore, cloud-based query evaluation raises two main challenge: (1) how to authenticate the query results in the format of key-value pairs; and (2) how to authenticate both the intermediate and final results for MapReduce execution. A few papers have considered integrity verification of MapReduce execution [34], [71], [74]. Intuitively, the intermediate results can be authenticated by applying the existing solutions on each mapper.…”

Section: Authentication Of Query Evaluation On Big Datamentioning

confidence: 99%

Integrity Authentication for SQL Query Evaluation on Outsourced Databases: A Survey

Zhang

Dong

Wang

2021

IEEE Trans. Knowl. Data Eng.

View full text Add to dashboard Cite

Spurred by the development of cloud computing, there has been considerable recent interest in the Database-as-a-Service (DaaS) paradigm. Users lacking in expertise or computational resources can outsource their data and database management needs to a third-party service provider. Outsourcing, however, raises an important issue of result integrity: how can the client verify with lightweight overhead that the query results returned by the service provider are correct (i.e., the same as the results of query execution locally)? This survey focuses on categorizing and reviewing the progress on the current approaches for result integrity of SQL query evaluation in the DaaS model. The survey also includes some potential future research directions for result integrity verification of the outsourced computations.Index Terms-Database-as-a-Service, SQL query, result integrity verification.1. Amazon Elastic Compute Cloud (EC2) [1] charges the users by the number of computing hours and the size of transferred data. It does not provide any detail of the involved computations (e.g., bandwidth, computing cycles, etc.) in the bills. Many online discussions [5], [6] have revealed that there are hidden service fees related to the computation details, e.g., how data is located on the servers.

show abstract

Section: Authentication Of Query Evaluation On Big Datamentioning

confidence: 99%

Integrity Authentication for SQL Query Evaluation on Outsourced Databases: A Survey

Zhang

Dong

Wang

2021

IEEE Trans. Knowl. Data Eng.

View full text Add to dashboard Cite

show abstract

“…In this group of malicious worker(s) detection approaches, a task is replicated or decomposed into pieces and the pieces are replicated [ 24 ]. Whenever the task queue is not empty, the master will pick one task and send it to any two workers.…”

Section: Background and Related Workmentioning

confidence: 99%

“…For detecting collusive workers, they improved the replicated task so that it is a credit-based replicated task, and they added a ` (trusted worker) to verify the intermediate results. In replication-based techniques, due to the task performed by multiple workers, system performance decreases [ 24 , 25 ].…”

Section: Background and Related Workmentioning

confidence: 99%

A Novel Hadoop Security Model for Addressing Malicious Collusive Workers

Sauber

Awad

Shawish

et al. 2021

Computational Intelligence and Neuroscience

View full text Add to dashboard Cite

With the daily increase of data production and collection, Hadoop is a platform for processing big data on a distributed system. A master node globally manages running jobs, whereas worker nodes process partitions of the data locally. Hadoop uses MapReduce as an effective computing model. However, Hadoop experiences a high level of security vulnerability over hybrid and public clouds. Specially, several workers can fake results without actually processing their portions of the data. Several redundancy-based approaches have been proposed to counteract this risk. A replication mechanism is used to duplicate all or some of the tasks over multiple workers (nodes). A drawback of such approaches is that they generate a high overhead over the cluster. Additionally, malicious workers can behave well for a long period of time and attack later. This paper presents a novel model to enhance the security of the cloud environment against untrusted workers. A new component called malicious workers’ trap (MWT) is developed to run on the master node to detect malicious (noncollusive and collusive) workers as they convert and attack the system. An implementation to test the proposed model and to analyze the performance of the system shows that the proposed model can accurately detect malicious workers with minor processing overhead compared to vanilla MapReduce and Verifiable MapReduce (V-MR) model [1]. In addition, MWT maintains a balance between the security and usability of the Hadoop cluster.

show abstract

“…In addition, the approach also depletes scalability. To lower the resource requirements, TrustMR [52], Trusted Samplingbased Third-party Result Verification (TS-TRV) [53], and Accountable MR [54] [55] are proposed. These schemes replicate only a subset of the tasks, thus reducing the resource consumption.…”

Section: B Mr Specific Solutionsmentioning

confidence: 99%

Communication Pattern Based Data Authentication (CPDA) Designed for Big Data Processing in a Multiple Public Cloud Environment

2020

View full text Add to dashboard Cite

With the development of cloud computing, there is a growing trend of multi-cloud Collaborative Big Data Computation (CBDC). In this environment, threats from authorized insiders are of particular concerns. Based on an extreme case of distributed computation where multiple collaborators jointly perform CBDC on shared datasets using an example distributed computing framework, MapReduce (MR), deployed in a Multiple Public Cloud (MPC) environment, this paper investigates how to protect the authenticity of data used during the computation in an efficient and scalable manner by proposing and evaluating a novel data authentication solution. The solution, called a Communication Pattern based Data Authentication (CPDA) framework, ensures data authenticity and non-repudiation of origin at the finest granularity without compromising efficiency and scalability. This is achieved by using an idea of communication pattern based authentication data aggregation. The framework has been comprehensively evaluated both theoretically and experimentally. The evaluation results show that the CPDA framework offers the strongest level of data authenticity protection (equivalent to that provided by digitally signing each data object individually) but introduces much lower overhead cost than the digital signature based solution. The results demonstrate that the idea of communication pattern based authentication data aggregation brings much benefit in terms of supporting efficient and scalable data authentication in a large-scale distributed system.

show abstract

TrustMR: Computation integrity assurance system for MapReduce

Cited by 17 publications

References 14 publications

Integrity Authentication for SQL Query Evaluation on Outsourced Databases: A Survey

Integrity Authentication for SQL Query Evaluation on Outsourced Databases: A Survey

A Novel Hadoop Security Model for Addressing Malicious Collusive Workers

Communication Pattern Based Data Authentication (CPDA) Designed for Big Data Processing in a Multiple Public Cloud Environment

Contact Info

Product

Resources

About