Wrapping a NoSQL Datastore for Stream Analytics

Mahmood, Khalid; Orsborn, Kjell; Risch, Tore

doi:10.1109/iri49571.2020.00050

Cited by 7 publications

(2 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…With the advent of the Industrial IoT (IIoT) and industrial analytics, many new application scenarios have emerged for which critical decisions rely on the large-scale analysis of sensor streams [32]. As data volumes grow exponentially, applications must handle larger amounts of unstructured and complex datasets simultaneously.…”

Section: Framework Designmentioning

confidence: 99%

A Distributed Real-Time Monitoring Scheme for Air Pressure Stream Data Based on Kafka

Zhou,

Chen

2024

Applied Sciences

View full text Add to dashboard Cite

Strict air pressure control is paramount in industries such as petroleum, chemicals, transportation, and mining to ensure production safety and to improve operational efficiency. In these fields, accurate real-time air pressure monitoring is critical to optimize operations and ensure facility and personnel safety. Although current Internet of Things air pressure monitoring systems enable users to make decisions based on objective data, existing approaches are limited by long response times, low efficiency, and inadequate preprocessing. Additionally, the exponential increase in data volumes creates the risk of server downtime. To address these challenges, this paper proposes a novel real-time air pressure monitoring scheme that uses Arduino microcontrollers in conjunction with GPRS network communication. It also uses Apache Kafka to construct a multi-server cluster for high-performance message processing. Furthermore, data are backed up by configuring multiple replications, which safeguards against data loss during server failures. The scheme also includes an intuitive and user-friendly visualization interface for data analysis and subsequent decision making. The experimental results demonstrate that this approach offers high throughput and timely responsiveness, providing a more reliable option for real-time gathering, analysis, and storage of massive data.

show abstract

Section: Framework Designmentioning

confidence: 99%

A Distributed Real-Time Monitoring Scheme for Air Pressure Stream Data Based on Kafka

Zhou,

Chen

2024

Applied Sciences

View full text Add to dashboard Cite

show abstract

“…Sensor data is typically a kind of stream data which can be stored in a NoSQL datastore for stream analytics (Mahmood, Orsborn, & Risch, 2020) and represented as time-series data. Some NoSQLbased storage models for massive time-series data have been developed (Di Martino et al, 2019).…”

Section: Related Workmentioning

confidence: 99%

An Efficient NoSQL-Based Storage Schema for Large-Scale Time Series Data

Ma,

Zhou,

2024

Journal of Database Management

View full text Add to dashboard Cite

In IoT (internet of things), most data from the connected devices change with time and have sampling intervals, which are called time-series data. It is challenging to design a time series storage model that can write massive time-series data in a short time and can query and analyze the persistent time-series data for a long time. This paper constructs the RHTSDB (Redis-HBase Time Series Database) storage model based on Redis and HBase. RHTSDB uses the memory database Redis (Remote Dictionary Server) to cache massive time-series data, providing efficient data storage and query functions. HBase is used in RHTSDB for long-term storage of time-series data to realize their persistence. The paper designs a cold and hot separation mechanism for time-series data, where the infrequently accessed cold data are stored in HBase, and the frequently accessed and latest data are stored in Redis. Experiments verify that RHTSDB has apparent advantages over Apache IoTDB and HBase in data intake and query efficiency.

show abstract