A data science challenge for converting airborne remote sensing data into ecological information

Abstract

Ecology has reached the point where data science competitions, in which multiple groups solve the same problem using the same data by different methods, will be productive for advancing quantitative methods for tasks such as species identification from remote sensing images. We ran a competition to help improve three tasks that are central to converting images into information on individual trees: (1) crown segmentation, for identifying the location and size of individual trees; (2) alignment, to match ground truthed trees with remote sensing; and (3) species classification of individual trees. Six teams (composed of 16 individual participants) submitted predictions for one or more tasks. The crown segmentation task proved to be the most challenging, with the highest-performing algorithm yielding only 34% overlap between remotely sensed crowns and the ground truthed trees. However, most algorithms performed better on large trees. For the alignment task, an algorithm based on minimizing the difference, in terms of both position and tree size, between ground truthed and remotely sensed crowns yielded a perfect alignment. In hindsight, this task was over simplified by only including targeted trees instead of all possible remotely sensed crowns. Several algorithms performed well for species classification, with the highest-performing algorithm correctly classifying 92% of individuals and performing well on both common and rare species. Comparisons of results across algorithms provided a number of insights for improving the overall accuracy in extracting ecological information from remote sensing. Our experience suggests that this kind of competition can benefit methods development in ecology and biology more broadly.

NEON NIST data science evaluation challenge: methods and results of team Conor

“…Alignment was performed between the provided ground and individual tree crown (ITC) datasets. The ITCs present in the dataset were divided up into training and test data as described in the parent paper (4). The ground data consisted of stem IDs, locations in latitude and longitude, stem heights, and stem diameters.…”

“…Species labels were also provided for each crown ID within the training set. Again, see (4) for more information on the input dataset.…”

“…These kinds of analysis are necessary for assessing the quality of a patch of habitat for conservation of particular target taxa(3)(8), for estimating system-level properties like primary productivity or capacity for carbon sequestration (11), and also for landowners interested in directly managing forests for wood or fruit production (10). Traditional methods of characterizing forests involved expensive, laborious, and time-consuming deployment of experts on foot to manually label individual trees with location, species, and structural data (1) (4). More recently, remote sensing technologies have emerged which show the potential to massively alter the scale and efficacy with which these characterizations can be performed.…”

“…Extracting useful ecosystem parameters from this mass of generated data involves three primary steps: segmentation, alignment, and classification (4). In the segmentation step, individual tree crowns are automatically extracted from the scene so that they can be counted and analyzed separately.…”

“…In general the efficacy of different remote sensing processes depends strongly on the forest type being surveyed -in particular the degree of canopy openness and overall species diversity (6). When new methods are introduced in the literature there is often a lack of robust comparison to existing methods, and the comparisons which are included are difficult to apply broadly due to these inherent differences in performance on different systems (4). As well, the formats in which remote sensing data are saved and processed vary hugely across platforms and research disciplines, and have proven difficult to standardize(4).…”