The DRIHM Project: A Flexible Approach to Integrate HPC, Grid and Cloud Resources for Hydro-Meteorological Research

D’Agostino, Daniele; Clematis, Andrea; Galizia, Antonella; Quarati, Alfonso; Danovaro, Emanuele; Roverelli, Luca; Zereik, Gabriele; Kranzlmüller, Dieter; Schiffers, Michael; Felde, Nils gentschen; Straube, Christian; Caumontz, Olivier; Richard, Évelyne; Garrote, Luís; Harpham, Quillon; Jagers, Bert; Dimitrijević, Vojin; Djekić, Ljiljana; Fiori, Elisabetta; Delogu, Fabio; Parodi, Antonio

doi:10.1109/sc.2014.49

Cited by 19 publications

(9 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This has to be assured for all alternative models ( Figure 2). Such arrangements have been supported by developing proper bridges implementing existing standards: NetCDF-CF 13 , WaterML 2.0 14 and OpenMI 15 . The process leading to the selection of the standards adopted in the project, and the detailed description of the bridge developed by the project are the subjects of a forthcoming HMR paper.…”

Section: A Model Interfaces and Standardizationmentioning

confidence: 99%

See 1 more Smart Citation

Setting Up an Hydro-Meteo Experiment in Minutes: The DRIHM e-Infrastructure for HM Research

Danovaro

Roverelli

Zereik

et al. 2014

2014 IEEE 10th International Conference on E-Science

View full text Add to dashboard Cite

Predicting weather and climate and its impacts on the environment, including hazards such as floods and landslides, is a big challenge that can be efficiently supported by a distributed and heterogeneous infrastructure, exploiting several kinds of computational resources: HPC, Grids and Clouds. This can help researchers in speeding up experiments, improve resolution and accuracy, simulate with different numerical models and model chains. Such numerical models are complex with heavy computational requirements, huge numbers of parameters to tune, and not fully standardized interfaces. Hence, each research entity is usually focusing on a limited set of tools and hard-wired solutions to enable their interaction. The DRIHM approach is based on strong standardization, well defined interfaces, and an easy to use web interface for model configuration and experiment definition. A researcher can easily compare outputs from different hydrologic models forced by the same meteorological model, or compare different meteorological models to validate or improve her research. This paper presents the benefit of a web-based interface for hydro-meteorology research through a detailed analysis of the portal (based on liferay-gUse) developed by the DRIHM project.

show abstract

Section: A Model Interfaces and Standardizationmentioning

confidence: 99%

“…The portal provides a seamless access to two classes of computing resources. The detailed and complete analysis of the DDCI, and the resource selection policy related to models submission can be found in [15]. In the following we briefly outline the main characteristics of the DDCI.…”

Section: B the Distributed Computing Infrastructurementioning

confidence: 99%

Setting Up an Hydro-Meteo Experiment in Minutes: The DRIHM e-Infrastructure for HM Research

Danovaro

Roverelli

Zereik

et al. 2014

2014 IEEE 10th International Conference on E-Science

View full text Add to dashboard Cite

show abstract

“…The Distributed Computing Infrastructure for Hydro‐Meteorology (DRIHM) project (D'Agostino et al ., , ; Danovaro et al ., ) is a European initiative aiming at providing an open, fully integrated eScience environment platform for predicting, managing, and mitigating the risks related to such extreme weather phenomena. It enables seamless access to a set of computing resources with the objective of providing a collection of services for performing experiments with numerical models in meteorology, hydrology, and hydraulics.…”

Section: Introductionmentioning

confidence: 99%

Using OpenMI and a Model MAP to Integrate WaterML2 and NetCDF Data Sources into Flood Modeling of Genoa, Italy

Harpham

Lhomme

Parodi

et al. 2016

J American Water Resour Assoc

Self Cite

View full text Add to dashboard Cite

Extreme hydrometeorological events such as flash floods have caused considerable loss of life and damage to infrastructure over recent years. Flood events in the Mediterranean region between 1990 and 2006 caused over 4,500 fatalities and cost over €29 billion in damage, with Italy one of the worst affected countries. The Distributed Computing Infrastructure for Hydro-Meteorology (DRIHM) project is a European initiative aiming at providing an open, fully integrated eScience environment for predicting, managing, and mitigating the risks related to such extreme weather phenomena. Incorporating both modeled and observational data sources, it enables seamless access to a set of computing resources with the objective of providing a collection of services for performing experiments with numerical models in meteorology, hydrology, and hydraulics. The purpose of this article is to demonstrate how this flexible modeling architecture has been constructed using a set of standards including the NetCDF and WaterML2 file formats, in-memory coupling with OpenMI, controlled vocabularies such as CF Standard Names, ISO19139 metadata, and a Model MAP (Metadata, Adaptors, Portability) gateway concept for preparing numerical models for standardized use. Hydraulic results, including the impact to buildings and hazards to people, are given for the use cases of the severe and fatal flash floods, which occurred in Genoa, Italy in November 2011 and October 2014.(KEY TERMS: computational methods; flooding; OpenMI; WaterML2; data management.)

show abstract

“…This work is complementary to three papers: the interfaces to abstract HM service provision from the underlying heterogeneous computing infrastructure composed of high-performance computing (HPC), Grid resources, and Cloud resources are presented in [4]; the usage of the portal for HM scientists in terms of the graphical user interfaces is described in [5]; and the validation of the science gateway is discussed in [6]. This work is complementary to three papers: the interfaces to abstract HM service provision from the underlying heterogeneous computing infrastructure composed of high-performance computing (HPC), Grid resources, and Cloud resources are presented in [4]; the usage of the portal for HM scientists in terms of the graphical user interfaces is described in [5]; and the validation of the science gateway is discussed in [6].…”

Section: Introductionmentioning

confidence: 99%

“…In this paper, we present the DRIHM portal. This work is complementary to three papers: the interfaces to abstract HM service provision from the underlying heterogeneous computing infrastructure composed of high-performance computing (HPC), Grid resources, and Cloud resources are presented in [4]; the usage of the portal for HM scientists in terms of the graphical user interfaces is described in [5]; and the validation of the science gateway is discussed in [6]. With respect to them, the present contribution is focused on the Information and Communications Technology (ICT) aspects related to the actual implementation of the portal using the grid and cloud User Support Environment / Web Services Parallel Grid Runtime and Developer Environment Portal science gateway toolkit [7] and the European e-infrastructures ecosystem.…”

Section: Introductionmentioning

confidence: 99%

Lessons learned implementing a science gateway for hydro‐meteorological research

D’Agostino

Danovaro

Clematis

et al. 2015

Concurrency and Computation

Self Cite

View full text Add to dashboard Cite

A full hydrometeorological (HM) simulation, from rainfall to impact on urban areas, is a multidisciplinary job, which relies on the execution of a workflow composed of complex and heterogeneous model engines. Moreover, the accuracy of the simulation is strongly dependent on an extensive set of configuration parameters, which have to be selected in a consistent way among the models. Within the Distributed Research Infrastructure for Hydro-Meteorology project, a Web-based science gateway was developed with the aim to support HM researchers in designing, executing, and managing HM experiments. The core of this science gateway is the portal, which takes care of generating all the configuration files and handles the execution of simulation steps on a heterogeneous computing infrastructure composed of high-performance computing, Grid resources, and Cloud resources. This paper presents technological insights about the implementation of the portal, with an analysis of the adopted technologies and infrastructures. Our experience highlights the need of coherent policies in the management of data, computational resources, and software components that represent the ecosystem to develop science gateways. D. D'AGOSTINO ET AL. ‡ ‡ The guidelines to contribute a new model is available at http://www.drihm.eu/index.php/support-centre/how-to-andfaqs/how-to-contribute-a-new-model-to-drihm. § § The Meteorological Archival and Retrieval System,

show abstract

The DRIHM Project: A Flexible Approach to Integrate HPC, Grid and Cloud Resources for Hydro-Meteorological Research

Cited by 19 publications

References 18 publications

Setting Up an Hydro-Meteo Experiment in Minutes: The DRIHM e-Infrastructure for HM Research

Setting Up an Hydro-Meteo Experiment in Minutes: The DRIHM e-Infrastructure for HM Research

Using OpenMI and a Model MAP to Integrate WaterML2 and NetCDF Data Sources into Flood Modeling of Genoa, Italy

Lessons learned implementing a science gateway for hydro‐meteorological research

Contact Info

Product

Resources

About