Sharing is caring? Measurement error and the issues arising from combining 3D morphometric datasets

Abstract

Geometric morphometrics is routinely used in ecology and evolution and morphometric datasets are increasingly shared among researchers, allowing for more comprehensive studies and higher statistical power (as a consequence of increased sample size). However, sharing of morphometric data opens up the question of how much nonbiologically relevant variation (i.e., measurement error) is introduced in the resulting datasets and how this variation affects analyses. We perform a set of analyses based on an empirical 3D geometric morphometric dataset. In particular, we quantify the amount of error associated with combining data from multiple devices and digitized by multiple operators and test for the presence of bias. We also extend these analyses to a dataset obtained with a recently developed automated method, which does not require human‐digitized landmarks. Further, we analyze how measurement error affects estimates of phylogenetic signal and how its effect compares with the effect of phylogenetic uncertainty. We show that measurement error can be substantial when combining surface models produced by different devices and even more among landmarks digitized by different operators. We also document the presence of small, but significant, amounts of nonrandom error (i.e., bias). Measurement error is heavily reduced by excluding landmarks that are difficult to digitize. The automated method we tested had low levels of error, if used in combination with a procedure for dimensionality reduction. Estimates of phylogenetic signal can be more affected by measurement error than by phylogenetic uncertainty. Our results generally highlight the importance of landmark choice and the usefulness of estimating measurement error. Further, measurement error may limit comparisons of estimates of phylogenetic signal across studies if these have been performed using different devices or by different operators. Finally, we also show how widely held assumptions do not always hold true, particularly that measurement error affects inference more at a shallower phylogenetic scale and that automated methods perform worse than human digitization.

Measurement error in μCT‐based three‐dimensional geometric morphometrics introduced by surface generation and landmark data acquisition

“…Other strategies to cope with inter‐observer error have been suggested in the literature (see Fruciano, and Fruciano et al. for a more detailed discussion).…”

“…The preservation and preparation of specimens can induce artefactual variance by altering the natural form of the structures of interest (Lee, 1982 [linear measurements]; Bonneau et al 2012). The variability within repeated measurements performed by the same observer and the variability between different observers can also contribute significantly to measurement error (Ross & Williams, 2008;Robinson & Terhune, 2017; reports of relatively large observer error without tests for statistical significance: Curth et al 2017;Fruciano et al 2017;Daboul et al 2018). Nevertheless, the magnitude of observer error often has been considered small or negligible compared with true biological variability (Richtsmeier et al 1995;O'Higgins & Jones, 1998;Lockwood et al 2002;Pujol et al 2014;Barbeito-Andr es et al 2016).…”

“…Studies on the effect of the choice of a method (including the choice of a device) for 3D landmark data acquisition have reported contradictory results on the significance of measurement error caused by method choice and on the effect of choice of method on observer error (e.g. Hale et al 2014;Fruciano et al 2017;Robinson & Terhune, 2017;Shearer et al 2017;Marcy et al 2018). Hale et al (2014), for example, found significant differences between landmark data directly digitized from the specimens and landmark data derived from computed tomography (CT) scans of the same specimens.…”

Tuatara and a new morphometric dataset for Rhynchocephalia: Comments on Herrera‐Flores et al.

“…) because combining datasets generated by different authors can introduce further confounding variation (Arnqvist & Mårtensson ; Fruciano ; Fruciano et al . ). The mixture of images used by Herrera‐Flores et al .…”

Low resolution scans provide a sufficiently accurate, cost- and time-effective alternative to high resolution scans for interspecific 3D shape analyses

“…Digitization error may also be reduced by taking averages of repeated 311 measurements (Arnqvist & Martensson 1998;). Third, using 3D scans also 312 introduces a source of systematic error relative to μCT scans, therefore we recommend not 313 combining them whenever possible (see also Fruciano et al 2017), and especially in studies on 314 small intra-specific variation. In summary, with a few precautions listed above, we expect that 315 for studies with similarly sized skulls or similarly low resolution scans, the variation due to error 316 will be sufficiently low for successful detection of interspecific shape differences.…”