IoT Big Data provenance scheme using blockchain on Hadoop ecosystem

Pajooh, Houshyar Honar; Rashid, M. A.; Alam, Fakhrul; Demidenko, Serge

doi:10.1186/s40537-021-00505-y

Cited by 28 publications

(9 citation statements)

References 38 publications

(36 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Figure 6 shows the MapReduce data flow, a model proposed by Google, which addresses a new programming paradigm for working with Big Data. This model allows the manipulation of Big Data in parallel and distributed way, in addition to providing fault tolerance, scaling Input/Output (I/O), and monitoring [18][19][20][21][22][23].…”

Section: The Seasonal Climate Predictionmentioning

confidence: 99%

“…6 The MapReduce Data Flow [18] down into lines; (3) Mapping-when each part (line) is computed for the key/value format; (4) Sort/Shuffle-when the Sort and Shuffle operations perform the sorting and grouping of data, according to the "key"; (5) Reducing-when calculating the values contained in each grouping; and, finally, (6) Output-when the result is recorded on the HDFS. Figure 7 shows the MapReduce flow applied to count words in a text [18][19][20][21][22][23].…”

Section: The Seasonal Climate Predictionmentioning

confidence: 99%

See 1 more Smart Citation

A canonical model for seasonal climate prediction using Big Data

et al. 2022

View full text Add to dashboard Cite

This article addresses the elaboration of a canonical model, involving methods, techniques, metrics, tools, and Big Data, applied to the knowledge of seasonal climate prediction, aiming at greater dynamics, speed, conciseness, and scalability. The proposed model was hosted in an environment capable of integrating different types of meteorological data and centralizing data stores. The seasonal climate prediction method called M-PRECLIS was designed and developed for practical application. The usability and efficiency of the proposed model was tested through a case study that made use of operational data generated by an atmospheric numerical model of the climate area found in the supercomputing environment of the Center for Weather Forecasting and Climate Studies linked to the Brazilian Institute for Space Research. The seasonal climate prediction uses ensemble members method to work and the main Big Data technologies used for data processing were: Python language, Apache Hadoop, Apache Hive, and the Optimized Row Columnar (ORC) file format. The main contributions of this research are the canonical model, its modules and internal components, the proposed method M-PRECLIS, and its use in a case study. After applying the model to a practical and real experiment, it was possible to analyze the results obtained and verify: the consistency of the model by the output images, the code complexity, the performance, and also to perform the comparison with related works. Thus, it was found that the proposed canonical model, based on the best practices of Big Data, is a viable alternative that can guide new paths to be followed.

show abstract

Section: The Seasonal Climate Predictionmentioning

confidence: 99%

Section: The Seasonal Climate Predictionmentioning

confidence: 99%

A canonical model for seasonal climate prediction using Big Data

et al. 2022

View full text Add to dashboard Cite

show abstract

“…Besides, it does not use a provenance model. Pajooh et al [54] detail a distributed data storage of a blockchain-enabled large-scale IoT system. The focus is on system performance.…”

Section: Systematic Literature Mappingmentioning

confidence: 99%

A Blockchain-Based Architecture for Trust in Collaborative Scientific Experimentation

et al. 2022

View full text Add to dashboard Cite

In scientific collaboration, data sharing, the exchange of ideas and results are essential to knowledge construction and the development of science. Hence, we must guarantee interoperability, privacy, traceability (reinforcing transparency), and trust. Provenance has been widely recognized for providing a history of the steps taken in scientific experiments. Consequently, we must support traceability, assisting in scientific results’ reproducibility. One of the technologies that can enhance trust in collaborative scientific experimentation is blockchain. This work proposes an architecture, named BlockFlow, based on blockchain, provenance, and cloud infrastructure to bring trust and traceability in the execution of collaborative scientific experiments. The proposed architecture is implemented on Hyperledger, and a scenario about the genomic sequencing of the SARS-CoV-2 coronavirus is used to evaluate the architecture, discussing the benefits of providing traceability and trust in collaborative scientific experimentation. Furthermore, the architecture addresses the heterogeneity of shared data, facilitating interpretation by geographically distributed researchers and analysis of such data. Through a blockchain-based architecture that provides support on provenance and blockchain, we can enhance data sharing, traceability, and trust in collaborative scientific experiments.

show abstract

“…However, as the network interlinks heterogeneous sources with correlating events, the risk of exposure to intruders becomes high [5]. So, ensuring the security of IoT network from attackers is a significant concern for IoT development [1,2,6].…”

Section: Introductionmentioning

confidence: 99%

eBF: An Enhanced Bloom Filter for Intrusion Detection in IoT

Gebretsadik

Nayak²,

Patgiri³

2022

Preprint

View full text Add to dashboard Cite

Intrusion detection is an essential process to identify malicious incidents and continuously alert the many users of the Internet of Things (IoT). The constant monitoring of events generated from more than millions of devices connected to the IoT and the extensive analysis of every event based on predefined security policies consumes enormous resources. Accordingly, performance enhancement is a crucial concern of intrusion detection in IoT and other massive Big Data Systems to ensure a secure environment efficiently. Like many Big Data systems, intrusion detection system of the IoT need to employ the fast membership filter, Bloom Filter, to quickly identify possible attacks. Bloom Filter is an admiringly fast and space-efficient data structure that quickly handle elements of extensive datasets in a small memory space. However, the trade-off between the query performance and the number of hash functions, and memory space and false positive probability remain issues of Bloom Filter. Thus, this article presents an enhanced Bloom Filter (eBF) that remarkably improves memory efficiency and introduces new techniques to accelerate the speed of search processing demand of intrusion detection system in IoT. We experimentally show the efficacy of eBF using real dataset of intrusion detection. The experimental result shows that the proposed filter is remarkably memory efficient, faster, and more accurate than the state-of-the-art filters. eBF requires 15x, 13x, and 8x less memory compared with Standard Bloom Filter, Cuckoo filter, and robustBF, respectively. Therefore, this new system will significantly impact the enhancement of the performance of intrusion detection of IoT that concurrently monitors several billion events crosschecking with the defined security policies.

show abstract

IoT Big Data provenance scheme using blockchain on Hadoop ecosystem

Cited by 28 publications

References 38 publications

A canonical model for seasonal climate prediction using Big Data

A canonical model for seasonal climate prediction using Big Data

A Blockchain-Based Architecture for Trust in Collaborative Scientific Experimentation

eBF: An Enhanced Bloom Filter for Intrusion Detection in IoT

Contact Info

Product

Resources

About