Data Integration and Analysis System (DIAS) as a Platform for Data and Model Integration: Cases in the Field of Water Resources Management and Disaster Risk Reduction

Kawasaki, Akiyuki; Koudelova, Petra; Tamakawa, Katsunori; Kitamoto, Asanobu; Ikoma, Eiji; Ikeuchi, Ken; Shibasaki, Ryosuke; Koike, Takao

doi:10.5334/dsj-2018-029

Cited by 5 publications

(1 citation statement)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This design results in a high performing, customizable, scalable, and resource-efficient weather analysis framework. Data Integration and Assimilation System (DIAS) [4] is another related system that provides efficient data storage, quality controls, sharing, and metadata management. Meteorological Assimilation Data Ingest System (MADIS) [5] enforces a standard data format and provides role-based data access.…”

Section: Introductionmentioning

confidence: 99%

WDIAS: A Microservices-Based Weather Data Integration and Assimilation System

Karunarathne

Bandara

Herath³

2020

2020 Moratuwa Engineering Research Conference (MERCon)

View full text Add to dashboard Cite

Numerical Weather Models (NWMs) utilize data from diverse sources such as automated weather stations, radars, and satellite images. Such multimodal data need to be transcoded into a NWM compatible format before use. Moreover, the data integration system's response time needs to be relatively low to reduce the time to forecast weather events like hurricanes and flash floods. Furthermore, the resulting data need to be accessed by many researchers and third-party applications. Existing weather data integration systems are based on monolithic or client-server architectures, and are proprietary or closed source. Hence, they are not only expensive to operate in an era of cloud computing, but also challenging to customize for regions with different weather patterns. In this paper, we present a Weather Data Integration and Assimilation System (WDIAS) that uses microservices architecture and container orchestration to achieve high scalability, availability, and low-cost operation. WDIAS provides a modular architecture to integrate data from different sources, enforce data quality controls, export data into different formats, and extend the functionality by adding new modules. Using a synthetic workload and an experimental setup on a public cloud, we demonstrate that WDIAS can handle 300 RPS with relatively low latency.

show abstract

Section: Introductionmentioning

confidence: 99%

WDIAS: A Microservices-Based Weather Data Integration and Assimilation System

Karunarathne

Bandara

Herath³

2020

2020 Moratuwa Engineering Research Conference (MERCon)

View full text Add to dashboard Cite

show abstract

A holistic approach for using global climate model (GCM) outputs in decision making

Illangasingha,

Koike,

Rasmy

et al. 2023

Journal of Hydrology

View full text Add to dashboard Cite

Technologies Enabling Situational Awareness During Disaster Response: A Systematic Review

Kedia

Ratcliff

O'Connor

et al. 2020

Disaster med. public health prep.

View full text Add to dashboard Cite

Situational awareness (SA) is critical to mobilizing a rapid, efficient, and effective response to disasters. Limited by time and resources, response agencies must make decisions about rapidly evolving situations, which requires the collection, analysis, and sharing of actionable information across a complex landscape. Emerging technologies, if appropriately applied, can enhance SA and enable responders to make quicker, more accurate decisions. The aim of this systematic review is to identify technologies that can improve SA and assist decision-making across the United States Government and the domestic and international agencies they support during disaster response operations. A total of 1459 articles and 36 after-action reports were identified during literature searches. Following the removal of duplicates and application of inclusion/exclusion criteria, 302 articles and after-action reports were included in the review. Our findings suggest SA is constrained primarily due to unreliable and significantly delayed communications, time-intensive data analysis and visualization, and a lack of interoperable sensor networks and other capabilities providing data to shared platforms. Many of these challenges could be addressed by existing technologies. Bridging the divide between research and development efforts and the operational needs of response agencies should be prioritized.

show abstract

Data Integration and Analysis System (DIAS) as a Platform for Data and Model Integration: Cases in the Field of Water Resources Management and Disaster Risk Reduction

Cited by 5 publications

References 23 publications

WDIAS: A Microservices-Based Weather Data Integration and Assimilation System

WDIAS: A Microservices-Based Weather Data Integration and Assimilation System

A holistic approach for using global climate model (GCM) outputs in decision making

Technologies Enabling Situational Awareness During Disaster Response: A Systematic Review

Contact Info

Product

Resources

About