Reducing Time to Results with EUMETSAT's New Data Services

Abstract

<p>EUMETSAT offers a vast and growing collection of earth observation data produced by over 35 years of operational meteorological satellites. New data products are produced 24/7x365 and consistency with previous satellites and other missions is ensured by intercalibration and reprocessing campaigns. The benefits for the geosciences community are readily apparent - a recent survey showed that EUMETSAT and its Satellite Application Facilities produce 26% of the Essential Climate Variable records identified by the Global Climate Observing System that can be observed from space.<br><br>With the advent of new core satellite programmes and many narrowly focused missions, the volume and complexity of the generated data products will increase significantly, making it unfeasible for traditional workflows, relying on accessing data holdings present on the user's premises, to fully exploit these observations.<br><br>Users can access EUMETSAT data via two service categories: “push” services, currently provided by EUMETCast Satellite and delivering data to users via satellite systems in near real-time, and “pull” services, currently provided by the Long Term Archive and by the EUMETSAT Visualisation Service (EUMETView). EUMETSAT is in the process of reshaping its data services portfolio by leveraging big data and cloud computing technologies. The new Data Services are being phased into operations during 2020 and address several challenges with using EUMETSAT's data: near real-time data access, accessing time series, viewing data, transforming it to make it compatible with downstream workflows, and processing data on the premises where they are stored.<br><br>EUMETSAT has established an on-premises hybrid cloud, in which new Data Services for online data access (Data Store), web map visualisations (View Service) and product format customisations (Data Tailor) are hosted. Additionally, our “push” services are extended, with the introduction of the EUMETCAST Terrestrial service.<br><br>The Data Store provides online access for directly downloading satellite data via a web-based user interface and APIs usable in processing chains. Users can download the data in its original format or customise it before download by invoking the Data Tailor Service. The View Service provides access via standard OGC Web Map, Web Coverage and Web Feature Services (WMS, WCS, WFS) which visualise data available in the Data Store. It is accessible via a web-based interface and APIs allowing the integration of visualisations in end-user applications. EUMETCast Terrestrial is an evolution of the EUMETCast Satellite system that relies on the network infrastructure provided by GEANT and its partners plus the Internet to deliver high volumes of data worldwide. EUMETCast Terrestrial is able to deliver data outside the EUMETCast Satellite footprint and to user communities large enough to benefit from a multicast service, but not large enough to justify a full satellite-based broadcast.<br><br>This presentation will showcase these new Data Services, which enable users to transition from traditional local data processing regimes to cloud-native research workflows. With the new Data Services, users can easily discover, explore, and tailor data products to their needs and thus shift the effort from data and infrastructure handling to domain-specific and scientific topics.</p>