Big data stream analysis: a systematic literature review

Abstract

Advances in information technology have facilitated large volume, high-velocity of data, and the ability to store data continuously leading to several computational challenges. Due to the nature of big data in terms of volume, velocity, variety, variability, veracity, volatility, and value [1] that are being generated recently, big data computing is a new trend for future computing. Big data computing can be generally categorized into two types based on the processing requirements, which are big data batch computing and big data stream computing Abstract Recently, big data streams have become ubiquitous due to the fact that a number of applications generate a huge amount of data at a great velocity. This made it difficult for existing data mining tools, technologies, methods, and techniques to be applied directly on big data streams due to the inherent dynamic characteristics of big data. In this paper, a systematic review of big data streams analysis which employed a rigorous and methodical approach to look at the trends of big data stream tools and technologies as well as methods and techniques employed in analysing big data streams. It provides a global view of big data stream tools and technologies and its comparisons. Three major databases, Scopus, ScienceDirect and EBSCO, which indexes journals and conferences that are promoted by entities such as IEEE, ACM, SpringerLink, and Elsevier were explored as data sources. Out of the initial 2295 papers that resulted from the first search string, 47 papers were found to be relevant to our research questions after implementing the inclusion and exclusion criteria. The study found that scalability, privacy and load balancing issues as well as empirical analysis of big data streams and technologies are still open for further research efforts. We also found that although, significant research efforts have been directed to real-time analysis of big data stream not much attention has been given to the preprocessing stage of big data streams. Only a few big data streaming tools and technologies can do all of the batch, streaming, and iterative jobs; there seems to be no big data tool and technology that offers all the key features required for now and standard benchmark dataset for big data streaming analytics has not been widely adopted. In conclusion, it was recommended that research efforts should be geared towards developing scalable frameworks and algorithms that will accommodate data stream computing mode, effective resource allocation strategy and parallelization issues to cope with the ever-growing size and complexity of data.

Task Allocation in Hybrid Big Data Analytics for Urban IoT Applications

“…In our current version, "batch" applications are executed in Hadoop MapReduce, and "stream" ones are in Storm. It comes from the safety fact that a single framework for either computing model can avoid the complexity of interoperability [Kolajo et al 2019]. By the allocator of coordinator, the application would be parsed as concrete tasks for allocation further.…”

Architectural Structures and Viewpoints of Streaming Big Data

“…Analysis data: utility to the aggregated data. The challenges show impact to storage capacity especially in data collection during generating the raw data which the streaming data treat with virtual memory (Buffer) [7]. On the other hand, the process to generate a dataset with clean data and setting the relationship need preprocess technique [8].…”

“…The authors in [7] found that scalability, privacy and stack adjusting issues, as well as an observational examination of huge data streams and innovations, are still open to encourage inquire about endeavours. They found that even though, noteworthy investigate endeavours have been directed to the real-time investigation of big data stream not much consideration has been given to the preprocessing arrange of enormous data streams.…”

Identifying Requirements for Big Data Analytics and Mapping to Hadoop Tools

“…Stream processing is a continuous process and it does not finish until stopped explicitly. Results of stream processing are readily accessible and will be repeatedly updated as new stream of data enter the system [40]. These systems can process nearly unbounded amount of data.…”