An Adaptable Seismic Data Format

Abstract

We present ASDF, the Adaptable Seismic Data Format, a modern and practical data format for all branches of seismology and beyond. The growing volume of freely available data coupled with ever expanding computational power opens avenues to tackle larger and more complex problems. Current bottlenecks include inefficient resource usage and insufficient data organization. Properly scaling a problem requires the resolution of both these challenges, and existing data formats are no longer up to the task. ASDF stores any number of synthetic, processed or unaltered waveforms in a single file. A key improvement compared to existing formats is the inclusion of comprehensive meta information, such as event or station information, in the same file. Additionally, it is also usable for any non-waveform data, for example, cross-correlations, adjoint sources or receiver functions. Last but not least, full provenance information can be stored alongside each item of data, thereby enhancing reproducibility and accountability. Any data set in our proposed format is self-describing and can be readily exchanged with others, facilitating collaboration. The utilization of the HDF5 container format grants efficient and parallel I/O operations, integrated compression algorithms and check sums to guard against data corruption. To not reinvent the wheel and to build upon past developments, we use existing standards like QuakeML, StationXML, W3C PROV and HDF5 wherever feasible. Usability and tool support are crucial for any new format to gain acceptance. We developed mature C/Fortran and Python based APIs coupling ASDF to the widely used SPECFEM3D_GLOBE and ObsPy toolkits.

Automated Large‐Scale Full Seismic Waveform Inversion for North America and the North Atlantic

“…Responding to this challenge, the focus of this work is the development of a scalable and automated full‐waveform inversion procedure. This includes various technical advances in the form of the ObsPy toolkit (Beyreuther et al, ; Krischer, Megies, et al, ; Megies et al, ) to more efficiently process data, the Large‐Scale Seismic Inversion Framework (LASIF; Krischer, Fichtner, et al, ) to aid in managing data in the context of full seismic waveform inversions, and the Adaptable Seismic Data Format (ASDF, Krischer et al, ) to support storing and exchanging data. A graph‐based workflow orchestration framework has been developed to couple these tools to a numerical optimization library resulting in an inversion procedure that can run without human intervention.…”

“…The black lines show the tectonic plate boundaries, and the hatched areas are orogens where a further microzonation in smaller tectonic plates is not performed. Plate boundaries and orogens are taken from Bird (2003). (LASIF; Krischer, Fichtner, et al, 2015) to aid in managing data in the context of full seismic waveform inversions, and the Adaptable Seismic Data Format (ASDF, Krischer et al, 2016) to support storing and exchanging data. A graph-based workflow orchestration framework has been developed to couple these tools to a numerical optimization library resulting in an inversion procedure that can run without human intervention.…”

A Data Quality Strategy to Enable FAIR, Programmatic Access across Large, Diverse Data Collections for High Performance Data Analysis

“…To do this, in consultation with the geophysics and HDF communities, the principles of the CF convention from the climate community and the ACDD from the Earth science community were translated into a proposed new geophysics convention that improves programmatic access and interoperability across different geophysical data types, such as seismic, gravity, magnetotelluric, radiometrics [18]. We also applied our benchmarking strategy to the geophysics domain, initially using the domain-popular ObsPy library [19] and SPECFEM3D code [20], to demonstrate how different organizations of the data (in terms of chunking size and compression) impact on the performance by comparing new data formats, such as PH5 [21] and ASDF [22] to traditional formats such as the Society of Exploration Geophysicists-Y Data Exchange Format (SEG-Y), the Standard for the Exchange of Earthquake Data Format (SEED), Seismic Analysis Code (SAC), etc.…”

ExSeisDat: A set of parallel I/O and workflow libraries for petroleum seismology

“…Open source formats, like HDF5 (Folk et al, 2011) and Adaptable Seismic Data Format (ASDF; Krischer et al, 2016), and proprietary formats, like SEIS and Geosoft Database, store metadata and trace data in a more contiguous manner. Typically, a seismic data processing program will instead work with its own proprietary format that's more easily parallelized, and expect a geophysicist to convert their trace data to a format supported by the program.…”