This is a continuous paper on limitations of commonly used metrics in image analysis. The current version discusses segmentation metrics only, while future versions will also include metrics for classification and detection tasks. For missing references, use cases, other comments or questions, please contact

Artificial Intelligence (AI) is having a tremendous impact across most areas of science. Applications of AI in healthcare have the potential to improve our ability to detect, diagnose, prognose, and intervene on human disease. For AI models to be used clinically, they need to be made safe, reproducible and robust, and the underlying software framework must be aware of the particularities (e.g. geometry, physiology, physics) of medical data being processed. This work introduces MONAI, a freely available, community-supported, and consortium-led PyTorch-based framework for deep learning in healthcare. MONAI extends PyTorch to support medical data, with a particular focus on imaging, and provide purpose-specific AI model architectures, transformations and utilities that streamline the development and deployment of medical AI models. MONAI follows best practices for software-development, providing an easy-to-use, robust, welldocumented, and well-tested software framework. MONAI preserves the simple, additive, and compositional approach of its underlying PyTorch libraries. MONAI is being used by and receiving contributions from research, clinical and industrial teams from around the world, who are pursuing applications spanning nearly every aspect of healthcare.

The field of automatic biomedical image analysis crucially depends on robust and meaningful performance metrics for algorithm validation. Current metric usage, however, is often ill-informed and does not reflect the underlying domain interest. Here, we present a comprehensive framework that guides researchers towards choosing performance metrics in a problem-aware manner. Specifically, we focus on biomedical image analysis problems that can be interpreted as a classification task at image, object or pixel level. The framework first compiles domain interest-, target structure-, data set-and algorithm output-related properties of a given problem into a problem fingerprint, while also mapping it to the appropriate problem category, namely image-level classification, semantic segmentation, instance segmentation, or object detection. It then guides users through the process of selecting and applying a set of appropriate validation metrics while making them aware of potential pitfalls related to individual choices. In this paper, we describe the current status of the Metrics Reloaded recommendation framework, with the goal of obtaining constructive feedback from the image analysis community. The current version has been developed within an international consortium of more than 60 image analysis experts and will be made openly available as a user-friendly toolkit after community-driven optimization.

Objective: Grey matter (GM) atrophy occurs in all multiple sclerosis (MS) phenotypes. We investigated whether there is a spatiotemporal pattern of GM atrophy that is associated with faster disability accumulation in MS. Disability was assessed with the Expanded-Disability Status Scale (EDSS). We obtained volumes of the deep GM (DGM), temporal, frontal, parietal, occipital and cerebellar GM, brainstem and cerebral white matter. Hierarchical mixed-models assessed annual percentage rate of regional tissue loss and identified regional volumes associated with time-to-EDSS progression. MethodsResults: SPMS showed the lowest baseline volumes of cortical GM and DGM. Of all baseline regional volumes, only that of the DGM predicted time-to-EDSS progression (hazard ratio=0.73, 95% CIs 0.65, 0.82; p<0.001): for every standard deviation decrease in baseline DGM volume, the risk of presenting a shorter time to EDSS worsening during follow-up increased by 27%. Of all longitudinal measures, DGM showed the fastest annual rate of atrophy, which was faster in SPMS (-1.45%), PPMS (-1.66%), and RRMS (-1.34%) than CIS (-0.88%) and HCs (-0.94%)[p<0.01]. The rate of temporal GM atrophy in SPMS (-1.21%) was significantly faster than RRMS (-0.76%), CIS (-0.75%), and HCs (-0.51%).Similarly, the rate of parietal GM atrophy in SPMS (-1.24-%) was faster than CIS (-0.63%) and HCs (-0.23%) (all p values <0.05). Only the atrophy rate in DGM in patients was significantly associated with disability accumulation (beta=0.04, p<0.001).. CC-BY-ND 4.0 International license It is made available under a (which was not peer-reviewed) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity.The copyright holder for this preprint . http://dx.doi.org/10.1101/182006 doi: bioRxiv preprint first posted online Aug. 29, 2017; 3 Interpretation: This large multi-centre and longitudinal study shows that DGM volume loss drives disability accumulation in MS, and that temporal cortical GM shows accelerated atrophy in SPMS than RRMS. The difference in regional GM atrophy development between phenotypes needs to be taken into account when evaluating treatment effect of therapeutic interventions.

Neuroimaging biomarkers play a prominent role for disease diagnosis or tracking neurodegenerative processes. Multiple methods have been proposed by the community to extract robust disease specific markers from various imaging modalities. Evaluating the accuracy and robustness of developed methods is difficult due to the lack of a biologically realistic ground truth.We propose a proof-of-concept method for a patient-and diseasespecific brain neurodegeneration simulator. The proposed scheme, based on longitudinal multi-modal data, has been applied to a population of normal controls and patients diagnosed with Alzheimer's disease or frontotemporal dementia. We simulated follow-up images from baseline scans and compared them to real repeat images. Additionally, simulated maps of volume change are generated, which can be compared to maps estimated from real longitudinal data. The results indicate that the proposed simulator reproduces realistic patient-specific patterns of longitudinal brain change for the given populations.

We present a framework for intrinsic comparison of surface metric structures and curvatures. This work parallels the work of Kurtek et al. on parameterization-invariant comparison of genus zero shapes. Here, instead of comparing the embedding of spherically parameterized surfaces in space, we focus on the first fundamental form. To ensure that the distance on spherical metric tensor fields is invariant to parameterization, we apply the conjugation-invariant metric arising from the L2 norm on symmetric positive definite matrices. As a reparameterization changes the metric tensor by a congruent Jacobian transform, this metric perfectly suits our purpose. The result is an intrinsic comparison of shape metric structure that does not depend on the specifics of a spherical mapping. Further, when restricted to tensors of fixed volume form, the manifold of metric tensor fields and its quotient of the group of unitary diffeomorphisms becomes a proper metric manifold that is geodesically complete. Exploiting this fact, and augmenting the metric with analogous metrics on curvatures, we derive a complete Riemannian framework for shape comparison and reconstruction. A by-product of our framework is a near-isometric and curvature-preserving mapping between surfaces. The correspondence is optimized using the fast spherical fluid algorithm. We validate our framework using several subcortical boundary surface models from the ADNI dataset.

Purpose Surveillance of patients with high-grade glioma (HGG) and identification of disease progression remain a major challenge in neurooncology. This study aimed to develop a support vector machine (SVM) classifier, employing combined longitudinal structural and perfusion MRI studies, to classify between stable disease, pseudoprogression and progressive disease (3-class problem). Methods Study participants were separated into two groups: group I (total cohort: 64 patients) with a single DSC time point and group II (19 patients) with longitudinal DSC time points (2-3). We retrospectively analysed 269 structural MRI and 92 dynamic susceptibility contrast perfusion (DSC) MRI scans. The SVM classifier was trained using all available MRI studies for each group. Classification accuracy was assessed for different feature dataset and time point combinations and compared to radiologists’ classifications. Results SVM classification based on combined perfusion and structural features outperformed radiologists’ classification across all groups. For the identification of progressive disease, use of combined features and longitudinal DSC time points improved classification performance (lowest error rate 1.6%). Optimal performance was observed in group II (multiple time points) with SVM sensitivity/specificity/accuracy of 100/91.67/94.7% (first time point analysis) and 85.71/100/94.7% (longitudinal analysis), compared to 60/78/68% and 70/90/84.2% for the respective radiologist classifications. In group I (single time point), the SVM classifier also outperformed radiologists’ classifications with sensitivity/specificity/accuracy of 86.49/75.00/81.53% (SVM) compared to 75.7/68.9/73.84% (radiologists). Conclusion Our results indicate that utilisation of a machine learning (SVM) classifier based on analysis of longitudinal perfusion time points and combined structural and perfusion features significantly enhances classification outcome (p value= 0.0001).

No abstract