A Fortran-Keras Deep Learning Bridge for Scientific Computing

Abstract

Implementing artificial neural networks is commonly achieved via high-level programming languages such as Python and easy-to-use deep learning libraries such as Keras. These software libraries come preloaded with a variety of network architectures, provide autodifferentiation, and support GPUs for fast and efficient computation. As a result, a deep learning practitioner will favor training a neural network model in Python, where these tools are readily available. However, many large-scale scientific computation projects are written in Fortran, making it difficult to integrate with modern deep learning methods. To alleviate this problem, we introduce a software library, the Fortran-Keras Bridge (FKB). This two-way bridge connects environments where deep learning resources are plentiful with those where they are scarce. The paper describes several unique features offered by FKB, such as customizable layers, loss functions, and network ensembles. The paper concludes with a case study that applies FKB to address open questions about the robustness of an experimental approach to global climate simulation, in which subgrid physics are outsourced to deep neural network emulators. In this context, FKB enables a hyperparameter search of one hundred plus candidate models of subgrid cloud and radiation physics, initially implemented in Keras, to be transferred and used in Fortran. Such a process allows the model’s emergent behavior to be assessed, i.e., when fit imperfections are coupled to explicit planetary-scale fluid dynamics. The results reveal a previously unrecognized strong relationship between offline validation error and online performance, in which the choice of the optimizer proves unexpectedly critical. This in turn reveals many new neural network architectures that produce considerable improvements in climate model stability including some with reduced error, for an especially challenging training dataset.

Combining data assimilation and machine learning to infer unresolved scale parametrization

“…multi-core supercomputers and graphics processing units). Training and running hybrid models efficiently imposes heavy requirements on both the hardware and software and may come with an overhead even if some tools are very promising [43].…”

WeatherBench: A Benchmark Data Set for Data‐Driven Weather Forecasting

“…First, the interaction between physical and machine learning components are poorly understood and can lead to unexpected instabilities and biases (Brenowitz & Bretherton, 2019). Second, they are difficult to implement from a technical perspective because one has to interface the machine learning components with complex climate model code, typically written in Fortran even though recent developments aim to make this step easier (Ott et al, 2020).…”

fv3gfs-wrapper: a Python wrapper of the FV3GFS atmospheric model

“…This approach also does not provide a way for Python to modify the behavior of the Fortran model. User code which modifies model behavior must be re-written in Fortran, for example when machine learning routines are trained in Python and exported to be used in a Fortran model (Ott et al, 2020;Curcic, 2019).…”

“…Solutions have so far been based on Fortran executables, either by calling Python from Fortran (Brenowitz and Bretherton, 2019) or by re-implementing neural network codes in Fortran (Ott et al, 2020;Curcic, 2019). Because of the strong tooling available for machine learning in Python, it is an advantage to be able to include Python machine learning code within the atmospheric model.…”