A CyberGIS-Jupyter Framework for Geospatial Analytics at Scale

Yin, Dandong; Liu, Yan Y.; Padmanabhan, Anand; Terstriep, Jeff; Rush, Johnathan; Wang, Shaowen

doi:10.1145/3093338.3093378

Cited by 18 publications

(17 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We will build an interactive methodology building and validation environment online using cyberGIS Jupyter (Yin et al. ) to further accelerate CFIM research, data and software integration, and computation.…”

Section: Discussionmentioning

confidence: 99%

“…continue to improve the usability of the CFIM computational framework to couple related hydrologic modeling processes for producing flood inundation forecasts at high spatial and temporal resolutions. We will build an interactive methodology building and validation environment online using cyberGIS Jupyter (Yin et al 2017) to further accelerate CFIM research, data and software integration, and computation.…”

Section: Journal Of the American Water Resources Associationmentioning

confidence: 99%

See 1 more Smart Citation

A CyberGIS Integration and Computation Framework for High‐Resolution Continental‐Scale Flood Inundation Mapping

Liu

Maidment²,

Tarboton

et al. 2018

J American Water Resour Assoc

Self Cite

View full text Add to dashboard Cite

We present a Digital Elevation Model‐based hydrologic analysis methodology for continental flood inundation mapping (CFIM), implemented as a cyberGIS scientific workflow in which a 1/3rd arc‐second (10 m) height above nearest drainage (HAND) raster data for the conterminous United States (CONUS) was computed and employed for subsequent inundation mapping. A cyberGIS framework was developed to enable spatiotemporal integration and scalable computing of the entire inundation mapping process on a hybrid supercomputing architecture. The first 1/3rd arc‐second CONUS HAND raster dataset was computed in 1.5 days on the cyberGIS Resourcing Open Geospatial Education and Research supercomputer. The inundation mapping process developed in our exploratory study couples HAND with National Water Model forecast data to enable near real‐time inundation forecasts for CONUS. The computational performance of HAND and the inundation mapping process were profiled to gain insights into the computational characteristics in high‐performance parallel computing scenarios. The establishment of the CFIM computational framework has broad and significant research implications that may lead to further development and improvement of flood inundation mapping methodologies.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Journal Of the American Water Resources Associationmentioning

confidence: 99%

A CyberGIS Integration and Computation Framework for High‐Resolution Continental‐Scale Flood Inundation Mapping

Liu

Maidment²,

Tarboton

et al. 2018

J American Water Resour Assoc

Self Cite

View full text Add to dashboard Cite

show abstract

“…The JupyterHub architecture forms the common technology platform underlying the web interface and data analysis components of the GEMS application. Use of this approach was readily accepted by the GEMS leadership in part because Jupyter has already been used many times in similar contexts [1,6,14].…”

Section: Science Portalmentioning

confidence: 99%

Jupyter as Common Technology Platform for Interactive HPC Services

Milligan

2018

Proceedings of the Practice and Experience on Advanced Research Computing

View full text Add to dashboard Cite

The Minnesota Supercomputing Institute has implemented Jupyterhub and the Jupyter notebook server as a general-purpose pointof-entry to interactive high performance computing services. This mode of operation runs counter to traditional job-oriented HPC operations, but offers significant advantages for ease-of-use, data exploration, prototyping, and workflow development. From the user perspective, these features bring the computing cluster nearer to parity with emerging cloud computing options. On the other hand, retreating from fully-scheduled, job-based resource allocation poses challenges for resource availability and utilization efficiency, and can involve tools and technologies outside the typical core competencies of a supercomputing center's operations staff. MSI has attempted to mitigate these challenges by adopting Jupyter as a common technology platform for interactive services, capable of providing command-line, graphical, and workflow-oriented access to HPC resources while still integrating with job scheduling systems and using existing compute resources. This paper will describe the mechanisms that MSI has put in place, advantages for research and instructional uses, and lessons learned.

show abstract

“…We made use of the Groups social collaboration, by making early versions of our research results available to invited participants of workshops and tutorial demonstrations to our Landlab group in HydroShare. The data and model are accessed by launching Jupyter Notebooks that access Landlab installed on JupyterHub servers at the National Center for Supercomputing Applications (Yin et al, 2017;Castranova, 2017). HydroShare features enable our current and future researchers to use the Copy Resource function to replicate our published resource (i.e., the landslide model) in their own account with Derived from metadata that reference back to the published resource DOI, to serve as a starting point for their work.…”

Section: Reproducibilitymentioning

confidence: 99%

A hydroclimatological approach to predicting regional landslide probability using Landlab

Strauch,

Istanbulluoglu,

Nudurupati

et al. 2018

HydroShare Resources

View full text Add to dashboard Cite

Abstract. We develop a hydroclimatological approach to the modeling of regional shallow landslide initiation that integrates spatial and temporal dimensions of parameter uncertainty to estimate an annual probability of landslide initiation based on Monte Carlo simulations. The physically based model couples the infinite-slope stability model with a steady-state subsurface flow representation and operates in a digital elevation model. Spatially distributed gridded data for soil properties and vegetation classification are used for parameter estimation of probability distributions that characterize model input uncertainty. Hydrologic forcing to the model is through annual maximum daily recharge to subsurface flow obtained from a macroscale hydrologic model. We demonstrate the model in a steep mountainous region in northern Washington, USA, over 2700 km 2 . The influence of soil depth on the probability of landslide initiation is investigated through comparisons among model output produced using three different soil depth scenarios reflecting the uncertainty of soil depth and its potential long-term variability. We found elevation-dependent patterns in probability of landslide initiation that showed the stabilizing effects of forests at low elevations, an increased landslide probability with forest decline at midelevations (1400 to 2400 m), and soil limitation and steep topographic controls at high alpine elevations and in post-glacial landscapes. These dominant controls manifest themselves in a bimodal distribution of spatial annual landslide probability. Model testing with limited observations revealed similarly moderate model confidence for the three hazard maps, suggesting suitable use as relative hazard products. The model is available as a component in Landlab, an open-source, Python-based landscape earth systems modeling environment, and is designed to be easily reproduced utilizing HydroShare cyberinfrastructure.

show abstract

A CyberGIS-Jupyter Framework for Geospatial Analytics at Scale

Cited by 18 publications

References 21 publications

A CyberGIS Integration and Computation Framework for High‐Resolution Continental‐Scale Flood Inundation Mapping

A CyberGIS Integration and Computation Framework for High‐Resolution Continental‐Scale Flood Inundation Mapping

Jupyter as Common Technology Platform for Interactive HPC Services

A hydroclimatological approach to predicting regional landslide probability using Landlab

Contact Info

Product

Resources

About