GODAE OceanView Class 4 forecast verification framework: global ocean inter-comparison

Abstract

As part of the work of the GODAE OceanView Inter-comparison and Validation Task Team (IV-TT), 6 global ocean forecasting systems spread across 5 operational oceanography forecast centres were inter-compared using a common set of observations as a proxy for the truth. The 'Class 4' in the title refers to a set of forecast verification metrics defined in the MERSEA-IP/GODAE internal metrics document (Hernandez 2007), the defining feature of which is that comparisons between forecasts and observations take place in observation space. This approach is seen as a departure from other diagnostic approaches such as analysing model trends or innovation statistics, and is commonly used in the atmospheric community. The physical parameters involved in the comparison are sea surface temperature (SST), sub-surface temperature, sub-surface salinity and sea level anomaly (SLA). SST was measured using in-situ observations obtained from USGODAE, sub-surface conditions were compared to Argo profiles, while sea level anomaly was measured by several satellite altimeters courtesy of AVISO. The 5 forecast centres involved in the project were Met Office, Australian Bureau of Meteorology, Mercator Océan, Environment Canada and NOAA/NWS/NCEP. Combining Met Office, Mercator Océan and Environment Canada forecasts into a mixed resolution multi-model ensemble produces estimates of the ocean state which have better accuracy and associativity properties for SST, SLA and temperature profiles than any individual ensemble component.

Recent progress in performance evaluations and near real-time assessment of operational ocean products

“…Scores are defined by RMSE, based on differences between observation and model values for each parameter normalized by the largest RMSE. Reference observations are fully described in the companion paper (Ryan et al 2015). Using this approach, one can characterize the relative score of each OOFS for each parameter.…”

“…This exercise also allows each of the partners to assess more carefully the forecast capability of every OOFS in the region of interest and measure the efficiency of each system. The Australian group has performed regionally this multi-system assessment presented in a second companion paper (Divakaran et al 2015).…”

“…Five OOCs are involved, and six OOFSs are compared, looking at SST, T/S at depth and SSH -with sea ice concentration in preparation. A companion paper (Ryan et al 2015) presents the global inter-comparison performed over basin scale areas. This exercise also allows each of the partners to assess more carefully the forecast capability of every OOFS in the region of interest and measure the efficiency of each system.…”

“…These issues are addressed in the framework of the ongoing GODAE OceanView program (Schiller et al 2015) (GOV) (www.godae-oceanview.org) by the Intercomparison and Validation Task Team (IV-TT). Three initiatives have started and that are ongoing: an Ocean Reanalysis Intercomparison (Balmaseda et al 2015); the organization of a multi-model ensemble forecast approach for ocean surface parameters; and the organization of the near-real-time operational product 'Class 4 metrics' inter-comparison against observations, described later in this paper and detailed in two companion papers (Ryan et al 2015;Divakaran et al 2015).…”

Forecasting the mixed-layer depth in the Northeast Atlantic: an ensemble approach, with uncertainties based on data from operational ocean forecasting systems

“…html. Intercomparisons are also regularly performed in the context of the GODAE Oceanview international initiative (Ryan et al, 2014). These evaluations based on direct comparisons with observations (assimilated and independent) show that the four systems' overall performance is state-ofthe-art.…”