Multivariate anomaly detection for Earth observations: a comparison of algorithms and feature extraction techniques

Abstract

Abstract. Today, many processes at the Earth's surface are constantly monitored by multiple data streams. These observations have become central to advancing our understanding of vegetation dynamics in response to climate or land use change. Another set of important applications is monitoring effects of extreme climatic events, other disturbances such as fires, or abrupt land transitions. One important methodological question is how to reliably detect anomalies in an automated and generic way within multivariate data streams, which typically vary seasonally and are interconnected across variables. Although many algorithms have been proposed for detecting anomalies in multivariate data, only a few have been investigated in the context of Earth system science applications. In this study, we systematically combine and compare feature extraction and anomaly detection algorithms for detecting anomalous events. Our aim is to identify suitable workflows for automatically detecting anomalous patterns in multivariate Earth system data streams. We rely on artificial data that mimic typical properties and anomalies in multivariate spatiotemporal Earth observations like sudden changes in basic characteristics of time series such as the sample mean, the variance, changes in the cycle amplitude, and trends. This artificial experiment is needed as there is no "gold standard" for the identification of anomalies in real Earth observations. Our results show that a well-chosen feature extraction step (e.g., subtracting seasonal cycles, or dimensionality reduction) is more important than the choice of a particular anomaly detection algorithm. Nevertheless, we identify three detection algorithms (k-nearest neighbors mean distance, kernel density estimation, a recurrence approach) and their combinations (ensembles) that outperform other multivariate approaches as well as univariate extreme-event detection methods. Our results therefore provide an effective workflow to automatically detect anomalies in Earth system science data.

Drought, Heat, and the Carbon Cycle: a Review

“…Carbon cycle impacts are not only determined or triggered by univariate climatological variables [3,4], and thus, various constellations of climate variables that lead to an extreme impact would have to be considered. These issues are currently discussed under the term Bcompound events^ [19,34] and, although still in its infancy, corresponding suitable multivariate detection methods are being developed [35][36][37]. As an alternative, definitions starting from extreme ecosystem or carbon cycle responses have been proposed [3].…”

Contrasting biosphere responses to hydrometeorological extremes: revisiting the 2010 western Russian heatwave

“…Summer heat and drought typically reduce (or even inhibit) photosynthesis, hence reducing the carbon uptake potential of ecosystems . However, the magnitude of these impacts varies between ecosystems (Frank et al, 2015), and the resulting net effects are still under debate, particularly for heatwaves . However, in-depth investigations of a number of individual events such as the European heatwave 2003 (Ciais et al, 2005), the 2000-2004 and 2012 droughts in North America (Schwalm et al, 2012;Wolf et al, 2016), and the RHW (Bastos et al, 2014) agree on an overall tendency towards negative impacts on the carbon accumulation potential.…”

Response to Reviewer 2

“…Many lines of evidence point towards an intensification of certain hydrometeorological extreme events, such as hot temperature extremes or droughts in many regions of the world over the next few decades (IPCC, 2012). Consequently, much research focuses on understanding how extreme hydrometeorological events affect ecosystems and their functioning (overviews of the state of research and concepts are given in, for example, Smith, 2011;Reyer et al, 2013;Niu et al, 2014;Frank et al, 2015). For instance, ecosystem responses could be manifested in extreme anomalies of phenology (Ma et al, 2015), biogeochemical fluxes (Frank et al, 2015), or even in altered ecosystem structure due to induced mortality (Hartmann et al, 2015).…”

“…Consequently, much research focuses on understanding how extreme hydrometeorological events affect ecosystems and their functioning (overviews of the state of research and concepts are given in, for example, Smith, 2011;Reyer et al, 2013;Niu et al, 2014;Frank et al, 2015). For instance, ecosystem responses could be manifested in extreme anomalies of phenology (Ma et al, 2015), biogeochemical fluxes (Frank et al, 2015), or even in altered ecosystem structure due to induced mortality (Hartmann et al, 2015). Global analyses of the geographical extent and integrated anomalies of extremes in the terrestrial biosphere reveal that only a very few extremes affect large areas, whereas most events are only of very local relevance (Reichstein et al, 2013).…”

“…Although EOs enable the detection of extremes in the terrestrial biosphere, a deeper understanding of impacts on ecosystem functioning can be gained from combining EOs with in situ observations (Frank et al, 2015;Babst et al, 2017). In fact, ecological in situ networks play an increasingly important role in analysing ecological phenomena and often provide a complementary perspective on natural phenomena to EOs (Nasahara and Nagai, 2015;Papale et al, 2015;Wingate et al, 2015) and complement model analyses (Rammig et al, 2015;Sippel et al, 2017).…”