Assessment of Despiking Methods for Turbulence Data in Micrometeorology

Abstract

The computation of turbulent fluxes of heat, momentum, and greenhouse gases requires measurements taken at high sampling frequencies. An important step in this process involves the detection and removal of sudden, short-lived variations that do not represent physical processes and that contaminate the data (i.e., spikes). The objective of this study is to assess the performance of several noteworthy despiking methodologies in order to provide a benchmark assessment and to provide a recommendation that is most applicable to high-frequency micrometeorological data in terms of efficiency and simplicity. The performance of a statistical time window–based algorithm widely used in micrometeorology is compared to three other methodologies (phase space, wavelet based, and median filter). These algorithms are first applied to a synthetic signal (a clean reference version and then one with spikes) in order to assess general performance. Afterward, testing is done on a time series of actual CO2 concentrations that contains extreme systematic spikes every hour owing to instrument interference, as well as several smaller random spike points. The study finds that the median filter and wavelet threshold methods are most reliable, and that their performance by far exceeds statistical time window–based methodologies that use the median or arithmetic mean operator (−34% and −71% reduced root-mean-square deviation, respectively). Overall, the median filter is recommended, as it is most easily automatable for a variety of micrometeorological data types, including data with missing points and low-frequency coherent turbulence.

Evapotranspiration in Northern Agro-Ecosystems: Numerical Simulation and Experimental Comparison

“…Sensible heat flux was measured locally (i.e., ecosystem scale) by means of and eddy-covariance (EC) instrument and processed considering signal distortions under all weather conditions [27] and, at landscape scale, based on a large aperture scintillometer (LAS) [28][29][30].…”

Computational efficiency for the surface renewal method

“…The example algorithms shown here improve or economize existing calculation methods, including despiking of time series data (Højstrup, 1993;Starkenburg et al, 2016), calculation of structure functions (Antonia and Van Atta, 1978), and Fourier analysis of signals, i.e., spectral analysis (Press, 2007;Stull, 1988). In each case, dramatically faster execution times were accomplished using simple programming improvements.…”

“…It is a common procedure when measuring environmental parameters, especially in challenging conditions or complex environments (Göckede et al, 2004;Starkenburg et al, 2016). The origin of spikes in a time series may be electronic or physical (sensor malfunction or actual physical non-errors); regardless of the origin, spikes can be recorded as abnormally large or small values, or may be marked by an error flag defined in the firmware.…”

“…Previous authors have described a variety of methods including use of autocorrelation (Højstrup, 1993) and statistics within a moving window (Vickers and Mahrt, 1997). A comprehensive review of despiking methods is presented by Starkenburg et al (2016), with emphasis on the accuracy and statistical robustness of different computation methods.…”

eddy4R 0.2.0: a DevOps model for community-extensible processing and analysis of eddy-covariance data based on R, Git, Docker, and HDF5

“…In Lee et al, 2015;Metzger et al, , 2013Metzger et al, , 2016Salmon et al, 2015;Serafimovich et al, 2013;Starkenburg et al, 2016;Vaughan et al, 2016;Xu et al, 2017). eddy4R currently consists of four packages: eddy4R.base, eddy4R.qaqc, eddy4R.turb, and eddy4R.erf.…”