The variability of crater identification among expert and community crater analysts

Abstract

Page 5 likely to yield the most reliable results. Kirchoff et al. (2011) provide a more recent comparison with three researchers (two expert, one novice without crater counting experience) from the same lab who used the same technique to identify, measure, and, in this case, classify craters by preservation state. They used Lunar Reconnaissance Orbiter Camera Wide-Angle Camera (LROC WAC) images of Mare Orientale. The two experienced analysts had counts that differed by 20-40% in a given diameter range, while the novice counter identified numerous features that are probably not craters, differing from the other two by >100% over some diameter ranges. They also had significant variation among the preservation states attributed to each crater, despite a relatively coarse fourpoint scale. This work showed that despite common thinking that crater counting is fairly easy and straightforward, there is a learning curve and an individual's crater counts should be discarded during the learning process. It also showed that even well defined crater morphologies may be difficult to classify uniformly. Hiesinger et al. (2012) also focused on lunar craters, in their case using LROC Narrow-Angle Camera (NAC) images at approximately 0.5 m/px. They were interested in reproducible results for better understanding the lunar cratering flux and performed a single test with two experienced researchers who used the same technique on the same image. The Heisinger et al.(2012) team found an overall variation of only ±2% between their analysts, a dispersion significantly less than previous work.What this brief review indicates is that while there has been some discussion in the literature about agreement between different researchers' crater identifications, (a) there has been no thorough discussion on researcher variability, (b) no published study discusses the variability when using different techniques for crater identification and measurement, (c) variation in crater morphology has not been discussed (e.g., sub-km craters appear substantially different at NAC pixel scales when compared with multi-km craters at WAC pixels scales), and (d) expert results have not been extensively compared with how well untrained or minimally trained crater counters do with the identification and measurement process. Given the proliferation of internet

Minimum effective area for high resolution crater counting of martian terrains

“…However, in the case of a noisy CTX image and/or for impact structures that are heavily degraded, a larger minimum diameter should be chosen to avoid resolution roll offs. Our choice of a 200 m minimum diameter avoids these issues and helps to mitigate human count errors (see Robbins et al (2014)), including count fatigue. Given the extreme sample size and diameter range of the analysis (>10 4 craters per map), more precise crater counting involving multi-point rim digitization or elliptical crater mapping was not conducted (Kneissl et al, 2011).…”

A Chang'E-1 global catalog of lunar impact craters

“…However, our system has identified 26351 new craters 43 km in diameter that are not listed in LU78287. As suggested by Robbins et al (2014), features consisting of at least 10 pixels are required to produce reliable and complete measurements; the greater number of craters guarantees that the catalog is an improvement over other records. An additional comparison of the results and an assessment of completeness was to take the size distribution of craters into account.…”

Estimating False Positive Contamination in Crater Annotations from Citizen Science Data

“…The software Application Programming Interface and database layer were developed by the Zooniverse team at Oxford University, building on their experience with storing and analysing large amounts of citizen science data (Lintott et al 2008). Moon Zoo was similar in design and aim to alternative projects including MoonMappers (Robbins et al 2014). One major goal of the Moon Zoo project was to gather crater statistics for the plotting of crater Size Frequency Distributions (SFDs).…”

“…The underlying Poisson assumption has become part of standard SFD analysis software (Michael and Neukum 2010). However, repeatability studies of experts and community crater counters (Robbins et al 2014) reveal uncertainties in counts far larger than those arising from Poisson perturbations alone. Earlier work has also shown subjective sources of uncertainty in both crater counts (Greely and Gault 1970;Kirchoff et al 2011) and size estimates (Gault 1970).…”

“…A simple approach to identifying false positives is to apply a threshold on the number of users who have annotated a particular crater. This assumes that multiple people are less likely to make the same misidentification (Robbins et al 2014). However, this approach requires a greater number of annotations to catch all craters, risking discarding real craters which have only been marked a small number of times.…”