Cloud-Native Repositories for Big Scientific Data

Abernathey, Ryan; Augspurger, Tom; Banihirwe, Anderson; Blackmon-Luca, Charles C.; Crone, T. J.; Gentemann, Chelle; Hamman, Joseph; Henderson, Naomi; Lepore, Chiara; McCaie, Theo; Robinson, Niall; Signell, Richard P.

doi:10.22541/au.160443768.88917719/v1

Cited by 10 publications

(11 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The nominal horizontal resolution of the ocean component of CM2.6 is 1/

normal10 °

, therefore resolving mesoscale eddies in many regions of the ocean (Hallberg, 2013). The data and tools for analysis were obtained from the Pangeo platform (Abernathey et al., 2021).…”

Section: Methodsmentioning

confidence: 99%

Stochastic‐Deep Learning Parameterization of Ocean Momentum Forcing

Guillaumin

Zanna

2021

J Adv Model Earth Syst

View full text Add to dashboard Cite

We use data from a realistic high-resolution coupled climate model to train a neu-6 ral network (NN) and perform offline tests 7 • We learn a stochastic parameterization of subgrid momentum forcing with the NN 8 that generalizes well to different regimes and climates 9 • The stochastic parameterization results in a stable implementation in an ideal-10 ized ocean model that improves the kinetic energy of the flow 11

show abstract

“…The nominal horizontal resolution of the ocean component of CM2.6 is 1/

normal10 °

, therefore resolving mesoscale eddies in many regions of the ocean (Hallberg, 2013). The data and tools for analysis were obtained from the Pangeo platform (Abernathey et al., 2021).…”

Section: Methodsmentioning

confidence: 99%

Stochastic‐Deep Learning Parameterization of Ocean Momentum Forcing

Guillaumin

Zanna

2021

J Adv Model Earth Syst

View full text Add to dashboard Cite

show abstract

“…The filter is isotropic, and uses a Gaussian target with a length scale of 300 km. The data are from a state‐of‐the‐art climate model, GFDL‐CM2.6 (Delworth et al., 2012; Griffies et al., 2015), obtained through the Pangeo cloud data library (Abernathey et al., 2021). The ocean component of GFDL‐CM2.6 utilizes the GFDL‐MOM5 numerical ocean code with a nominal resolution of 0.1°.…”

Section: Illustrative Examplesmentioning

confidence: 99%

“…Figure 10 illustrates the application of our filters to a single 5‐day average of AVISO estimates of absolute geostrophic velocity on a 0.25° grid obtained from Copernicus European Earth Observation program (https://marine.copernicus.eu) via Pangeo (Abernathey et al., 2021). The upper left panel shows the unfiltered surface kinetic energy defined as

false| boldu {false|}^{2} / 2

.…”

Section: Illustrative Examplesmentioning

confidence: 99%

Diffusion‐Based Smoothers for Spatial Filtering of Gridded Geophysical Data

Grooms

Loose

Abernathey

et al. 2021

J Adv Model Earth Syst

Self Cite

View full text Add to dashboard Cite

We describe a new way to apply a spatial filter to gridded data from models or observations, focusing on low‐pass filters. The new method is analogous to smoothing via diffusion, and its implementation requires only a discrete Laplacian operator appropriate to the data. The new method can approximate arbitrary filter shapes, including Gaussian filters, and can be extended to spatially varying and anisotropic filters. The new diffusion‐based smoother's properties are illustrated with examples from ocean model data and ocean observational products. An open‐source Python package implementing this algorithm, called gcm‐filters, is currently under development.

show abstract

“…The open science community Pangeo Project (http://pangeo.io/) (Abernathey et al., 2017) is creating a “Pangeo Forge,” to crowdsource and automate the creation of cloud optimized data (https://github.com/pangeo-forge/roadmap). This change in how data are accessed is due to improvements in the software used to access data (Abernathey et al., 2020).…”

Section: Innovations In Datamentioning

confidence: 99%

Science Storms the Cloud

et al. 2021

Self Cite

View full text Add to dashboard Cite

The core tools of science (data, software, and computers) are undergoing a rapid and historic evolution, changing what questions scientists ask and how they find answers. Earth science data are being transformed into new formats optimized for cloud storage that enable rapid analysis of multi‐petabyte data sets. Data sets are moving from archive centers to vast cloud data storage, adjacent to massive server farms. Open source cloud‐based data science platforms, accessed through a web‐browser window, are enabling advanced, collaborative, interdisciplinary science to be performed wherever scientists can connect to the internet. Specialized software and hardware for machine learning and artificial intelligence are being integrated into data science platforms, making them more accessible to average scientists. Increasing amounts of data and computational power in the cloud are unlocking new approaches for data‐driven discovery. For the first time, it is truly feasible for scientists to bring their analysis to data in the cloud without specialized cloud computing knowledge. This shift in paradigm has the potential to lower the threshold for entry, expand the science community, and increase opportunities for collaboration while promoting scientific innovation, transparency, and reproducibility. Yet, we have all witnessed promising new tools which seem harmless and beneficial at the outset become damaging or limiting. What do we need to consider as this new way of doing science is evolving?

show abstract

Cloud-Native Repositories for Big Scientific Data

Cited by 10 publications

References 0 publications

Stochastic‐Deep Learning Parameterization of Ocean Momentum Forcing

Stochastic‐Deep Learning Parameterization of Ocean Momentum Forcing

Diffusion‐Based Smoothers for Spatial Filtering of Gridded Geophysical Data

Science Storms the Cloud

Contact Info

Product

Resources

About