As part of the work of the GODAE OceanView Inter-comparison and Validation Task Team (IV-TT), 6 global ocean forecasting systems spread across 5 operational oceanography forecast centres were inter-compared using a common set of observations as a proxy for the truth. The 'Class 4' in the title refers to a set of forecast verification metrics defined in the MERSEA-IP/GODAE internal metrics document (Hernandez 2007), the defining feature of which is that comparisons between forecasts and observations take place in observation space. This approach is seen as a departure from other diagnostic approaches such as analysing model trends or innovation statistics, and is commonly used in the atmospheric community. The physical parameters involved in the comparison are sea surface temperature (SST), sub-surface temperature, sub-surface salinity and sea level anomaly (SLA). SST was measured using in-situ observations obtained from USGODAE, sub-surface conditions were compared to Argo profiles, while sea level anomaly was measured by several satellite altimeters courtesy of AVISO. The 5 forecast centres involved in the project were Met Office, Australian Bureau of Meteorology, Mercator Océan, Environment Canada and NOAA/NWS/NCEP. Combining Met Office, Mercator Océan and Environment Canada forecasts into a mixed resolution multi-model ensemble produces estimates of the ocean state which have better accuracy and associativity properties for SST, SLA and temperature profiles than any individual ensemble component.

Operational oceanography can be described as the provision of routine oceanographic information needed for decision-making purposes. It is dependent upon sustained research and development through the end-to-end framework of an operational service, from observation collection to delivery mechanisms. The core components of operational oceanographic systems are a multi-platform observation network, a data management system, a data assimilative prediction system, and a dissemination/accessibility system. These are interdependent, necessitating communication and exchange between them, and together provide the mechanism through which a clear picture of ocean conditions, in the past, present, and future, can be seen. Ocean observations play a critical role in all aspects of operational oceanography, not only for assimilation but as part of the research cycle, and

Operational oceanography is now established in many countries, focusing on global, regional, or coastal areas, and targeting different aspects of the « blue », « white » or « green » ocean processes in order to provide reliable information to users. There are nowadays a large variety of interests and users, with different disciplines and levels of expertise. Validation and verification of operational products and systems are evolving in order to anticipate user's needs, and better quantify the level of confidence on all these variety of ocean products. Operational oceanography evaluation development is in front of key issues: Ocean models are reaching the submesoscale description, which is currently not adequately observed; many products are available now for a given ocean variable, and often discrepancies are larger than similarities; real time forecasting systems are also challenged by reanalyses or reprocessed time series; operational systems are getting more complex, with coupled modelling, where errors from the different compartment need to be carefully addressed in order to measure their performance and provide further improvements. In parallel, the global ocean observing system is continuously completed with additional satellites in the constellation, with innovative sensors on new satellite missions, with efforts to better integrate the global, regional and coastal in-situ observing capabilities, and the design of new instrument, like the BGC-Argo that should bring an enhanced description of the ocean biogeochemical variability. This book chapter provides an overview of the existing, mature, validation and verification science in operational oceanography; discusses the ongoing efforts and new strategies; presents some of the structured groups and outcomes; and lists a series of challenges on the field.

Abstract. Ocean monitoring and forecasting services are increasingly being used by a diverse community of public and commercial organizations. The Met Office, as the body responsible for severe weather prediction, has for many years been involved in providing forecasts of aspects of the marine environment. This paper describes how these have evolved to include a range of wave, surge, and ocean reanalysis, analysis, and forecasts services. To support these services, and to ensure they evolve to meet the demands of users and are based on the best available science, a number of scientific challenges need to be addressed. The paper goes on to summarize the key challenges, and highlights some priorities for the ocean monitoring and forecasting research group at the Met Office. There is a need to both develop the underpinning science of the modelling and data assimilation systems and to maximize the benefits from observations and other inputs to the systems. Systematic evaluation underpins this science, and also needs to be the focus of research.

Abstract. The Met Office currently runs two operational ocean forecasting configurations for the North West European Shelf: an eddy-permitting model with a resolution of 7 km (AMM7) and an eddy-resolving model at 1.5 km (AMM15). Whilst qualitative assessments have demonstrated the benefits brought by the increased resolution of AMM15, particularly in the ability to resolve finer-scale features, it has been difficult to show this quantitatively, especially in forecast mode. Applications of typical assessment metrics such as the root mean square error have been inconclusive, as the high-resolution model tends to be penalised more severely, referred to as the double-penalty effect. This effect occurs in point-to-point comparisons whereby features correctly forecast but misplaced with respect to the observations are penalised twice: once for not occurring at the observed location, and secondly for occurring at the forecast location, where they have not been observed. An exploratory assessment of sea surface temperature (SST) has been made at in situ observation locations using a single-observation neighbourhood-forecast (SO-NF) spatial verification method known as the High-Resolution Assessment (HiRA) framework. The primary focus of the assessment was to capture important aspects of methodology to consider when applying the HiRA framework. Forecast grid points within neighbourhoods centred on the observing location are considered as pseudo ensemble members, so that typical ensemble and probabilistic forecast verification metrics such as the continuous ranked probability score (CRPS) can be utilised. It is found that through the application of HiRA it is possible to identify improvements in the higher-resolution model which were not apparent using typical grid-scale assessments. This work suggests that future comparative assessments of ocean models with different resolutions would benefit from using HiRA as part of the evaluation process, as it gives a more equitable and appropriate reflection of model performance at higher resolutions.

No abstract

Abstract. Ocean monitoring and forecasting services are increasingly being used by a diverse community of public and commercial organisations. The Met Office, as the body responsible for severe weather prediction, has for many years been involved in providing forecasts of aspects of the marine environment. This paper describes how these have evolved to include a range of wave, surge and ocean reanalysis, analysis and forecasts services. To support these services, and to ensure they evolve to meet the demands of users and are based on the best available science, a number of scientific challenges need to be addressed. The paper goes on to summarise the key challenges, and highlights some priorities for the ocean monitoring and forecasting research group at the Met Office. There is a need to both develop the underpinning science of the modelling and data assimilation systems and to maximise the benefits from observations and other inputs to the systems. Systematic evaluation underpins this science, and also needs to be the focus of research.

<div> <p>The importance of oceans for atmospheric forecasts as well as climate simulations is being increasingly recognised with the advent of coupled ocean / atmosphere forecast models. Having comparable resolutions in both domains maximises the benefits for a given computational cost. The Met Office has recently upgraded its operational global ocean-only model from an eddy permitting 1/4 degree tripolar grid (ORCA025) to the eddy resolving 1/12 degree ORCA12 configuration while retaining 1/4 degree data assimilation. </p> </div><div> <p>We will present a description of the ocean-only ORCA12 system, FOAM-ORCA12, alongside some initial results. Qualitatively, FOAM-ORCA12 seems to represent better (than FOAM-ORCA025) the details of mesoscale features in SST and surface currents. Overall, traditional statistical results suggest that the new FOAM-ORCA12 system performs similarly or slightly worse than the pre-existing FOAM-ORCA025. However, it is known that comparisons of models running at different resolutions suffer from a double penalty effect, whereby higher-resolution models are penalised more than lower-resolution models for features that are offset in time and space. Neighbourhood verification methods seek to make a fairer comparison using a common spatial scale for both models and it can be seen that, as neighbourhood sizes increase, ORCA12 consistently has lower continuous ranked probability scores (CRPS) than ORCA025. CRPS measures the accuracy of the pseudo-ensemble created by the neighbourhood method and generalises the mean absolute error measure for deterministic forecasts. </p> </div><div> <p>The focus over the next year will be on diagnosing the performance of both the model and assimilation. A planned development that is expected to enhance the system is the update of the background-error covariances used for data assimilation. </p> </div>