O2A: A generic framework for enabling the flow of sensor observations to archives and publications

Abstract

Abstract.Over the last two decades, the Alfred Wegener Institute (AWI) has been continuously committing to develop and sustain an e-Infrastructure for coherent discovery, visualization, dissemination and archival of scientific information in polar and marine regions. Most of the data originates from research activities being carried out in a wide range of AWI-operated research platforms: vessels, land-based stations, ocean-based stations and aircrafts. Archival and publishing in PANGAEA repository along with DOI assignment to individual datasets is a typical end-of-line step for most data owners.Within AWI, a workflow for data acquisition from vessel-mounted devices along with ingestion procedures for the raw data into the institutional archives has been well-established for many years. However, the increasing number of ocean-based stations and respective sensors along with heterogeneous project-driven requirements towards satellite communication, sensor monitoring, QA/QC control and validation, processing algorithms, visualization and dissemination has recently lead us to build a more generic and cost-effective framework. This framework, hereafter named O2A, has as main strength its seamless flow of sensor observation to archives and the fact that it complies with internationally used OGC standards and thus assuring interoperability in international context (e.g. SOS/SWE, WPS, WMS WFS,..).O2A is comprised of several extensible and exchangeable modules (e.g. controlled vocabularies and gazetteers, file type and structure validation, aggregation solutions, processing algorithms, etc) as well as various interoperability services. At the first data tier level, not only each sensor is being described following SensorML data model standards but the data is being fed to an SOS interface offering streaming solutions along with support to O&M encoding. Project administrators or data specialists are now able to monitor the individual sensors displayed in a map by simply clicking on the station and viewing the near real-time data for the selected station and sensor. In addition, the monitoring dashboards we built provide assistance to data scientists and administrators in terms of early detection of malfunction of sensors (e.g., email/SMS notification), filtering of data values for certain range (e.g. temperature values above a certain range) and data aggregation (e.g. calculation of daily averages).

Standardized monitoring of permafrost thaw: a user-friendly, multiparameter protocol

“…Our metadata protocol complies with the standards of the Open Geospatial Consortium (OGC) (Open Geospatial Consortium, 2021) and thus facilitates interoperability. Specifically, everything related to data processing and data management follows Observation to Archive (O2A; Koppe et al 2015;Gerchow et al 2017) and in turn all instrumentation parts of O2A follow sensorML specification (OGC, 2014).…”

“…It must follow the semantics of a standardized, controlled vocabulary to have broadly applicable language for machine access and processing. We apply the O2A dataflow framework which includes the comprehensive description and management of all data with metadata, central data storage and controlled data access (Koppe et al 2015;Gerchow et al 2017).…”

The Digital Earth Smart Monitoring Concept and Tools

“…Both, standardized procedures in the data acquisition (monitoring set-up, acquisition, calibration, data cleaning,) and standardized metadata, decide on the usability and trustworthiness of monitoring data. Cooperation with existing initiatives and infrastructures such as DataFlow Framework from Sensor Observations to Archives (O2A at AWI) and Modular Observation Solutions for Earth Systems (MOSES) were intensified to improve existing tools and bundle competencies (UFZ 2021; Koppe et al 2015;Gerchow et al 2015).…”

Ocean Data Product Integration Through Innovation-The Next Level of Data Interoperability

“…Most importantly however, user engagement also encompasses the responsibility to ensure that data are correctly understood by different end users. This makes it necessary to monitor, document and archive (using standardized metadata protocols) all elements of the data lifecycle, from sampling protocols via the properties, precision and accuracy of different sensors to archiving in accepted repositories such as Pangaea or EMODnet's GEOSS Portal, and to make relevant metadata available in a wellorganized and transparent form relevant to potential end users (Koppe et al, 2015).…”