No abstract

This is a continuous paper on limitations of commonly used metrics in image analysis. The current version discusses segmentation metrics only, while future versions will also include metrics for classification and detection tasks. For missing references, use cases, other comments or questions, please contact

The field of automatic biomedical image analysis crucially depends on robust and meaningful performance metrics for algorithm validation. Current metric usage, however, is often ill-informed and does not reflect the underlying domain interest. Here, we present a comprehensive framework that guides researchers towards choosing performance metrics in a problem-aware manner. Specifically, we focus on biomedical image analysis problems that can be interpreted as a classification task at image, object or pixel level. The framework first compiles domain interest-, target structure-, data set-and algorithm output-related properties of a given problem into a problem fingerprint, while also mapping it to the appropriate problem category, namely image-level classification, semantic segmentation, instance segmentation, or object detection. It then guides users through the process of selecting and applying a set of appropriate validation metrics while making them aware of potential pitfalls related to individual choices. In this paper, we describe the current status of the Metrics Reloaded recommendation framework, with the goal of obtaining constructive feedback from the image analysis community. The current version has been developed within an international consortium of more than 60 image analysis experts and will be made openly available as a user-friendly toolkit after community-driven optimization.

ORCID(s): 0000-0003-4910-9368 (L. Maier-Hein); 0000-0002-0713-8761 (M. Eisenmann); 0000-0002-2083-4999 (D. Sarikaya); 0000-0003-3503-1918 (K. März); 0000-0002-9441-8306 (T. Collins); 0000-0001-9477-9403 (A. Malpani); 0000-0003-0964-4497 (H. Feussner); 0000-0002-8745-1343 (S. Giannarou); 0000-0001-7288-3023 (P. Mascagni); 0000-0002-0121-2828 (H. Nakawala); 0000-0001-9139-8082 (C. Pugh); 0000-0002-0980-3227 (D. Stoyanov); 0000-0001-9809-4823 (K. Cleary); 0000-0002-6354-262X (G. Fichtinger); 0000-0002-4960-7554 (G. Forestier); 0000-0002-1772-4887 (B. Gibaud); 0000-0002-5140-0982 (M. Hashizume); 0000-0002-8371-760X (S. Onogur); 0000-0001-6272-1100 (R. Taylor); 0000-0002-9831-9110 (M. Wagner); 0000-0002-5010-4137 (N. Padoy); 0000-0002-7415-071X (P. Jannin);

About 50% of prostate cancer (PCa) tumors are TMPRSS2:ERG (T2E) fusion-positive (T2E+), but the role of T2E in PCa progression is not fully understood. We were interested in investigating epigenomic alterations associated with T2E+ PCa. Using different sequencing cohorts, we found several transcripts of the miR-449 cluster to be repressed in T2E+ PCa. This repression correlated strongly with enhanced expression of NOTCH and several of its target genes in TCGA and ICGC PCa RNA-seq data. We corroborated these findings using a cellular model with inducible T2E expression. Overexpression of miR-449a in vitro led to silencing of genes associated with NOTCH signaling (NOTCH1, HES1) and HDAC1. Interestingly, HDAC1 overexpression led to the repression of HES6, a negative regulator of the transcription factor HES1, the primary effector of NOTCH signaling, and promoted cell proliferation by repressing the cell cycle inhibitor p21. Inhibition of NOTCH as well as knockdown of HES1 reduced the oncogenic properties of PCa cell lines. Using tissue microarray analysis encompassing 533 human PCa cores, ERG-positive areas exhibited significantly increased HES1 expression. Taken together, our data suggest that an epigenomic regulatory network enhances NOTCH signaling and thereby contributes to the oncogenic properties of T2E+ PCa.