Training a supermodel with noisy and sparse observations: a case study with CPT and the synch rule on SPEEDO - v.1

Abstract

Abstract. In alternative to using the standard multi-model ensemble (MME) approach to combine the output of different models to improve prediction skill, models can also be combined dynamically to form a so-called supermodel. The supermodel approach allows for a quicker correction of the model errors. In this study we focus on weighted supermodels, in which the supermodel state is a weighted superposition of different imperfect model states. The estimation, “the training”, of the optimal weights of this combination is a critical aspect in the construction of a supermodel. In our previous works two algorithms were developed: (i) cross pollination in time (CPT-based technique), and, (ii) a synchronization based learning rule (synch rule). Those algorithms have been so far applied under the assumption of complete and noise-free observations. Here we go beyond and consider the more realistic case of noisy data that do not cover the full system's state and are not taken at each model's computational time step. We revise the training methods to cope with this observational scenario, while still being able to estimate accurate weights. In the synch rule an additional term is introduced to maintain physical balances, while in CPT nudging terms are added to let the models stay closer to the observations during training. Furthermore, we propose a novel formulation of the CPT method allowing for the weights to be negative. This makes it possible for CPT to deal with cases in which the individual model biases have the same sign, a situation that hampers constructing a skilful weighted supermodel based on positive weights. With these developments, both CPT and the synch rule have been made suitable to train a supermodel consisting of state-of-the-art weather or climate models.

Framework for an Ocean‐Connected Supermodel of the Earth System

“…However, the single model to which we connect can vary spatially and for different variables (Du & Smith, 2017;Schevenhoven & Carrassi, 2021;Smith, 2001). Furthermore, if one can afford an ensemble of supermodels (with several members for each model), models could be synchronised from a randomly drawn single member/model every time, so that the frequency of the optimal weight is satisfied.…”

“…Models are connected as they run via their state variables or their tendencies. Models can either be connected to each other (e.g., Mirchev et al, 2012;Smith, 2001) or toward their weighted mean (Schevenhoven & Carrassi, 2021;Wiegerinck et al, 2013). During a training phase, the connection terms are optimized to formulate a new synchronised dynamical system that achieves enhanced performance.…”

“…Supermodeling was successfully demonstrated for parametric model error forming a convex envelope around the truth and that with low dimensional dynamical systems (e.g., Lorenz 63, Lorentz 96, Rossler systems; see Mirchev et al, 2012;Van den Berge et al, 2011;Du & Smith, 2017), quasi-geostrophic atmospheric models Wiegerinck & Selten, 2017), and the global atmosphere-ocean-land model of intermediate complexity SPEEDO Selten et al, 2017). However, when the model error does not cancel out (i.e., parameters do not form a convex envelop around the truth), one can use negative weights (Schevenhoven & Carrassi, 2021;Schevenhoven et al, 2019), which raises new challenges. Furthermore, the supermodel can degrade performance on time scales different to those it was trained on (Wiegerinck & Selten, 2017).…”

A multi-model ensemble Kalman filter for data assimilation and forecasting

“…Synchronization-based supermodels nudge each model towards the others by coupling the evolution equations [22,23]. CPT and synchronizationbased supermodels were compared in [24,25]. Several other methods were compared for meteorological applications in 26 and 27.…”

“…Previously, MM-DA was only tested on very lowdimensional models with non-chaotic behavior [16,36], and recursive multi-step forecasts were not tested. Methods for multi-model forecasting have often been tested with perfect observations for calibration, single forecasts for each model rather than ensembles, and models that all share the same space [21,24]; several papers, though, have extended this work to noisy observations [20,25]. Here, we conduct experiments of the proposed method for both DA and forecasting in various settings, including models of different dimensionality and different-sized ensembles.…”