Technology advances have created a wide variety of novel, inexpensive sensors in the millimeter range that can be attached to or embedded into smartphones. These sensors are now directly connected to the Internet enabling us to collect high frequency updates from potentially thousands of mobile sensors densely deployed over an urban area. Today, data stream management systems (DSMS) are powerful data processing tools for update rates of 100,000-500,0000 tuples/s. In this paper, we investigate extending DSMS for monitoring continuous environmental phenomena such as air borne toxins or air quality based on up to 250K individual mobile sensor updates per query window to be spatially interpolated into a smooth, grid-based representation in near real-time. We propose a stream query operator approach and investigate different strategies to achieve near real-time spatial interpolation, while investigating memory footprint, runtime efficiency and interpolation quality of the different strategies.
Technological advances have created an unprecedented availability of inexpensive sensors capable of streaming environmental data in real-time. Data stream engines (DSE) with tuple processing rates of around 500k tuples/s have demonstrated their ability to both keep up with large numbers of spatio-temporal data streams, and execute stream window queries over them efficiently. Typically, geographically distributed sensors take samples asynchronously; however, when approximating the reality of a continuous phenomenon -such as the radiation field over an urban region-the objective is to integrate their values correctly over space as well as over time. This paper presents an approach to extend DSEs with support enabling sliding window queries over dynamic continuous phenomena, which return both spatio-temporal snapshot and movies as window query results. We introduce a novel grid-pane index as a main memory index structure shared between multi-queries over a phenomenon and an adaptive, data driven kNN algorithm for efficiently approximating cells based on available stream data samples. AkNN implements a spatio-temporal inverse distance weighting interpolation (IDW) method that integrates time with space via an anisotropic ratio. Further, we introduce the shell list template that allows quick calculation of NN cells by distance in a space-time (ST) cuboid. We performed extensive performance evaluations using the Fukushima nuclear event in March 2011 as a test data set.
With advances in technology and an increasing variety of inexpensive geosensors, environmental monitoring has become increasingly sensor dense and real time. Using sensor data streams enables real‐time applications such as environmental hazard detection, or earthquake, wildfire, or radiation monitoring. In‐depth analysis of such spatial fields is often based on a continuous representation. With very large numbers of concurrent observation streams, novel algorithms are necessary that integrate streams into rasters, or other continuous representations, continuously in real time. In this article, we present an approach leveraging data stream engines (DSEs) to achieve scalable, high‐throughput inverse distance weighting (IDW). In detail, we designed and implemented a novel stream query operator framework that extends general‐purpose DSEs. The proposed framework includes a two‐panel, spatio‐temporal grid‐based index and several algorithms, namely the Shell and k‐Shell algorithms, to estimate individual grid cells efficiently and adaptively for different sampling scenarios. For our performance experiments, we generated several different spatio‐temporal stream data sets based on the radiation deposits in the Fukushima region after the nuclear accident of 2011 in Japan. Our results showed that the k‐Shell algorithm of the proposed framework produces a raster based on 250k observation streams in under 0.5 s using a state‐of‐the‐art workstation.
ABSTRACT:With live streaming sensors and sensor networks, increasingly large numbers of individual sensors are deployed in physical space. Sensor data streams are a fundamentally novel mechanism to deliver observations to information systems. They enable us to represent spatio-temporal continuous phenomena such as radiation accidents, toxic plumes, or earthquakes almost as instantaneously as they happen in the real world. Sensor data streams discretely sample an earthquake, while the earthquake is continuous over space and time. Programmers attempting to integrate many streams to analyze earthquake activity and scope need to write code to integrate potentially very large sets of asynchronously sampled, concurrent streams in tedious application code. In previous work, we proposed the field stream data model (Liang et al., 2016) for data stream engines. Abstracting the stream of an individual sensor as a temporal field, the field represents the Earth's movement at the sensor position as continuous. This simplifies analysis across many sensors significantly. In this paper, we undertake a feasibility study of using the field stream model and the open source Data Stream Engine (DSE) Apache Spark(Apache Spark, 2017) to implement a real-time earthquake event detection with a subset of the 250 GPS sensor data streams of the Southern California Integrated GPS Network (SCIGN). The field-based real-time stream queries compute maximum displacement values over the latest query window of each stream, and related spatially neighboring streams to identify earthquake events and their extent. Further, we correlated the detected events with an USGS earthquake event feed. The query results are visualized in real-time.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.
hi@scite.ai
10624 S. Eastern Ave., Ste. A-614
Henderson, NV 89052, USA
Copyright © 2024 scite LLC. All rights reserved.
Made with 💙 for researchers
Part of the Research Solutions Family.