Shape analysis, a field in need of careful validation

Abstract

In the last two decades, the statistical analysis of shape has become an actively studied field and finds applications in a wide range of areas. In addition to algorithmic development, many researchers have distributed end-user orientated toolboxes, which further enable the utilization of the algorithms in an "off the shelf" fashion. However, there is little work on the evaluation and validation of these techniques, which poses a rather serious challenge when interpreting their results. To address this lack of validation, we design a validation framework and then use it to test some of the most widely used toolboxes. Our initial results show inconsistencies and disagreement among four different methods. We believe this type of analysis to be critical not only for the community of algorithm designers but also perhaps more importantly to researchers who use these tools without knowing the algorithm details and seek objective criteria for tool selection.

Shape analysis based on depth-ordering

“…When measuring the detected significant region in the local analysis results, the ROD is defined as the region whose FDR-corrected p -value is less than or equal to 0.05. The area ratio reported in Gao et al (2014) is 0.61 for SPHARM-PDM, 0 for ShapeWorks, and 0.17 for TBM.…”

“…Compared to the qualitative results of the three methods presented in Gao et al (2014), our method provides more accurate locations of the detected deformations, according to the introduced deformations. Furthermore, in contrast to these methods, our results reveal that most of the significantly different regions are located on the medial side of the putamen, instead of the lateral side, which is also consistent with the synthetic data.…”

“…In Gao et al (2014), three standard methods, SPHARM-PDM, Shapeworks, and Tensor Based Morphometry (TBM), were tested on the same synthetic dataset that we are using in the experiments. To quantitatively measure the significantly deformed region, they extracted the “region of deformation” (ROD) from ground truth data, and computed the ratio between the area of deformation and the area of the entire shape.…”

O5‐01‐02: Differential Hippocampal Shapes in Posterior Cortical Atrophy Patients: A Comparison With Control and Typical Ad Subjects

“…Given that the hippocampi are relatively small structures it may be that using higher resolution scans would enable the detection of more subtle shape differences between groups. Finally, caution is required when interpreting the results from shape analysis studies-a recent study indicated that the SPHARM-PDM method of shape analysis might overestimate regions of significant difference [Gao et al, 2014]. We used stringent statistical methods (family-wise error correction) in order to minimize false detection of differences where there were in fact none.…”

Measures of Morphological Complexity of Gray Matter on Magnetic Resonance Imaging for Control Age Grouping

“…It is known that GM is an important evidence for the assessment of cerebral atrophy revealed by MRI scans of the brain, because GM regions are found to be specifically susceptible to the cognitive decline and AD process [22]. Furthermore, the importance of the measurement of structures and changes of the brain during its development, aging, learning, disease and evolution is recognized as an emerging field of neuro-informatics known as brain morphometry, which aims to quantify anatomical features of the brain in terms of shape, mass, and volume using typically MRI data to gain insight into the pattern of the brain and its longitudinal extent across individuals or different biological species [23].…”