The Broken Window: An algorithm for quantifying and characterizing misleading trajectories in ecological processes

Abstract

<jats:title>Abstract</jats:title><jats:p>A fundamental problem in ecology is understanding how to scale discoveries: from patterns we observe in the lab or the plot to the field or the region or bridging between short term observations to long term trends. At the core of these issues is the concept of trajectory—that is, when can we have reasonable assurance that we know where a system is going? In this paper, we describe a <jats:italic>non-random resampling</jats:italic> method to directly address the temporal aspects of scaling ecological observations by leveraging existing data. Findings from long-term research sites have been hugely influential in ecology because of their unprecedented longitudinal perspective, yet short-term studies more consistent with typical grant cycles and graduate programs are still the norm.</jats:p><jats:p>We directly address bridging the gap between the short-term and the long-term by developing an automated, systematic resampling approach: in short, we repeatedly ‘sample’ moving windows of data from existing long-term time series, and analyze these sampled data as if they represented the entire dataset. We then compile typical statistics used to describe the relationship in the sampled data, through repeated samplings, and then use these derived data to gain insights to the questions: 1) <jats:italic>how often are the trends observed in short-term data misleading, and</jats:italic> 2) <jats:italic>can we use characteristics of these trends to predict our likelihood of being misled?</jats:italic> We develop a systematic resampling approach, the ‘bad-breakup’ algorithm, and illustrate its utility with a case study of firefly observations produced at the Kellogg Biological Station Long-Term Ecological Research Site (KBS LTER). Through a variety of visualizations, summary statistics, and downstream analyses, we provide a standardized approach to evaluating the trajectory of a system, the amount of observation required to find a meaningful trajectory in similar systems, and a means of evaluating our confidence in our conclusions.</jats:p>

How long do population level field experiments need to be? Utilising data from the 40‐year‐old LTER network

“…Next, to understand the study duration needed to identify changes in effect size over time for each dataset, we applied a moving window algorithm developed in R (Bahlai et al . 2020). We fit linear regression models to defined subsets of each dataset.…”

“…See Bahlai et al . 2020 for an in‐depth description of the process and github (https://github.com/cbahlai/broken_window) for the appropriate R code.…”

“…As such, research conducted at constrained time scales has the potential to be misleading, either capturing short‐term trends or failing to detect trends at all (Bahlai et al . 2020; Cusser et al . 2020).…”

Experimenting With the Past to Improve Environmental Monitoring

“…With non-random sampling, we learn about the elements of a good monitoring program by examining which subsamples of the data are most influential and the number of subsamples needed to have a high probability of detecting the true value of the metric. Bahlai et al (2020) describes this technique from a computational viewpoint specifically for time series. Although this approach has been used previously (Grantham et al, 2008;Bennett et al, 2016;Wauchope et al, 2019;White, 2019;Bahlai et al, 2020;Cusser et al, 2020), its adoption has been largely informal and not specifically stated.…”

Longer study length, standardized sampling techniques, and broader geographic scope leads to higher likelihood of detecting stable abundance patterns in long term black-legged tick studies

“…To analyze the stability of the abundance trends of these datasets, we used the ‘broken window’ algorithm in R (Bahlai 2020). This non-random resampling approach uses subsamples of the time series data to gain insights into patterns (Bahlai et al 2020). This algorithm defines the linear slope associated with the longest time series available as the ‘true’ slope (as it is calculated using the largest possible sample size) and performs calculations relative to this proxy for truth.…”