Neuroimage signature from salient keypoints is highly specific to individuals and shared by close relatives

Abstract

Neuroimaging studies typically adopt a common feature space for all data, which may obscure aspects of neuroanatomy only observable in subsets of a population, e.g. cortical folding patterns unique to individuals or shared by close relatives. Here, we propose to model individual variability using a distinctive keypoint signature: a set of unique, localized patterns, detected automatically in each image by a generic saliency operator. The similarity of an image pair is then quantified by the proportion of keypoints they share using a novel Jaccard-like measure of set overlap. Experiments demonstrate the keypoint method to be highly efficient and accurate, using a set of 7536 T1-weighted MRIs pooled from four public neuroimaging repositories, including twins, non-twin siblings, and 3334 unique subjects. All same-subject image pairs are identified by a similarity threshold despite confounds including aging and neurodegenerative disease progression. Outliers reveal previously unknown data labeling inconsistencies, demonstrating the usefulness of the keypoint signature as a computational tool for curating large neuroimage datasets.

Efficient Pairwise Neuroimage Analysis Using the Soft Jaccard Index and 3D Keypoint Sets

“…As keypoint sets are variable sized, typical metrics based on fixed-length vectors such as L-norms [5], [6] do not readily apply. Distances defined based on the Jaccard index or intersection-over-union [8] have proven to be effective in recent studies investigating genetics and brain MRI [9], [10]. For example, by defining set equivalence in terms of nearest neighbor (NN) descriptor correspondences, the Jaccard distance may be used to predict pairwise relationships.…”

“…A novel kernel is introduced that normalizes variability in pairwise keypoint displacement by the geometrical mean of keypoint scales, accounting for localization uncertainty in scale-space. This work extends keypoint-based neuroimaging analysis methods based on hard set equivalence (HSE) and appearance descriptors [9], [10], [13].…”

GPU optimization of the 3D Scale-invariant Feature Transform Algorithm and a Novel BRIEF-inspired 3D Fast Descriptor

“…For example, the ability to identify and characterize individuals and family members in large sets 3D scans is crucial in achieving accurate, personalized healthcare and minimizing patient-level labeling errors. Currently the 3D SIFT-Rank keypoint indexing (Toews and Wells III, 2013) is the only method reporting this capability (Chauvin et al, 2020(Chauvin et al, , 2021.…”

“…diffusion MRI histograms of the human brain (Chauvin et al, 2018). The Jaccard distance between feature sets was introduced to characterize the bag-of-feature manifold, to automatically flag errors in large public MRI datasets (Chauvin et al, 2019(Chauvin et al, , 2020, and most recently the first approach to identify family members from brain MRI (Chauvin et al, 2021).…”

Curating Subject ID Labels using Keypoint Signatures

“…due to double-dipping) or potential patient misdiagnosis in clinical contexts. Here we describe a highly efficient system for curating subject ID labels in large generic medical image datasets, based on the 3D image keypoint representation, which recently led to the discovery of previously unknown labeling errors in widely-used public brain MRI datasets [1].…”