Environmental Monitoring 2.0

Michel, Sebastian; Salehi, Ali; Luo, Lijuan; Dawes, Nicholas; Aberer, Karl; Barrenetxea, Guillermo; Bavay, Mathias; Kansal, Aman; Kumar, K. Kiran; Nath, Suman; Parlange, Marc B.; Tansley, Stewart; Ingen, Catharine van; Zhao, Feng; Zhou, Yongluan

doi:10.1109/icde.2009.49

Cited by 16 publications

(15 citation statements)

References 3 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In the project, environmental scientists are deploying a number of sensor stations to study the environmental changes and to provide alerts if needed (e.g., avalanche alert, etc). An on-going effort of the project is to share the sensor data to other scientists and to the public over the world via Microsoft's SensorMap [25]. This potentially requires the processing of a huge number of real-time stream queries.…”

Section: Motivating Scenariomentioning

confidence: 99%

See 1 more Smart Citation

Scalable delivery of stream query result

2009

Self Cite

View full text Add to dashboard Cite

Continuous queries over data streams typically produce large volumes of result streams. To scale up the system, one should carefully study the problem of delivering the result streams to the end users, which, unfortunately, is often overlooked in existing systems. In this paper, we leverage Distributed Publish/Subscribe System (DPSS), a scalable data dissemination infrastructure, for efficient stream query result delivery. To take advantage of DPSS's multicast-like data dissemination architecture, one has to exploit the common contents among different result streams and maximize the sharing of their delivery. Hence, we propose to merge the user queries into a few representative queries whose results subsume those of the original ones, and disseminate the result streams of these representative queries through the DPSS. To realize this approach, we study the stream query containment theories and propose efficient query grouping and merging algorithms. The proposed approach is nonintrusive and hence can be easily implemented as a middleware to be incorporated into existing stream processing systems. A prototype is developed on top of an open-source stream processing system and results of an extensive performance study on real datasets verify the efficacy of the proposed techniques.

show abstract

Section: Motivating Scenariomentioning

confidence: 99%

“…To copy otherwise, or to republish, to post on servers or to redistribute to lists, requires a fee and/or special permission from the publisher, ACM. VLDB '09, August [24][25][26][27][28]2009, Lyon, France Copyright 2009 VLDB Endowment, ACM 000-0-00000-000-0/00/00.…”

Section: Introductionmentioning

confidence: 99%

Scalable delivery of stream query result

2009

Self Cite

View full text Add to dashboard Cite

show abstract

“…This makes it possible for a casual developer to readily develop his own plug-in, supporting his own data source, with very little overhead. In its current version MeteoIO includes plug-ins for reading and/or writing time series and/or grids from Oracle and PostgreSQL databases, the Global Sensor Network (GSN) REST API (Michel et al, 2009), Cosmo XML (cos, 2013, GRIB, NetCDF, ARC ASCII, ARPS, GRASS, PGM, PNG, GEOtop, Snowpack and Alpine3D native file formats and a few others.…”

Section: Data Readingmentioning

confidence: 99%

MeteoIO 2.4.2: a preprocessing library for meteorological data

Bavay¹,

2014

View full text Add to dashboard Cite

Abstract.Using numerical models which require large meteorological data sets is sometimes difficult and problems can often be traced back to the Input/Output functionality. Complex models are usually developed by the environmental sciences community with a focus on the core modelling issues. As a consequence, the I/O routines that are costly to properly implement are often error-prone, lacking flexibility and robustness. With the increasing use of such models in operational applications, this situation ceases to be simply uncomfortable and becomes a major issue.The MeteoIO library has been designed for the specific needs of numerical models that require meteorological data. The whole task of data preprocessing has been delegated to this library, namely retrieving, filtering and resampling the data if necessary as well as providing spatial interpolations and parameterizations. The focus has been to design an Application Programming Interface (API) that (i) provides a uniform interface to meteorological data in the models, (ii) hides the complexity of the processing taking place, and (iii) guarantees a robust behaviour in the case of format errors, erroneous or missing data. Moreover, in an operational context, this error handling should avoid unnecessary interruptions in the simulation process.A strong emphasis has been put on simplicity and modularity in order to make it extremely easy to support new data formats or protocols and to allow contributors with diverse backgrounds to participate. This library is also regularly evaluated for computing performance and further optimized where necessary. Finally, it is released under an Open Source license and is available at http://models.slf.ch/p/meteoio. This paper gives an overview of the MeteoIO library from the point of view of conceptual design, architecture, features and computational performance. A scientific evaluation of the produced results is not given here since the scientific algorithms that are used have already been published elsewhere.

show abstract

“…In this example, the actual reading at t = 5 at the producer node is assumed to exceed a given error bound from its corresponding model-driven value generated by the preset model, requiring the producer node to reconstruct the model as M. 2 . The producer node then finds the most similar preset parameter values (i.e., 0.0 and 5 in Fig.…”

Section: A Overviewmentioning

confidence: 99%

“…In the SwissEx project [1], [2], for example, various research institutes share real-time environmental observation data across many different local sensor networks; they become producers and consumers of streaming data simultaneously. In these federated sensor networks, data streams from a producer node are continuously delivered to multiple consumer nodes in different local networks.…”

Section: Introductionmentioning

confidence: 99%