Going to Extremes: Weakly Supervised Medical Image Segmentation

Abstract

Medical image annotation is a major hurdle for developing precise and robust machine-learning models. Annotation is expensive, time-consuming, and often requires expert knowledge, particularly in the medical field. Here, we suggest using minimal user interaction in the form of extreme point clicks to train a segmentation model which, in effect, can be used to speed up medical image annotation. An initial segmentation is generated based on the extreme points using the random walker algorithm. This initial segmentation is then used as a noisy supervision signal to train a fully convolutional network that can segment the organ of interest, based on the provided user clicks. Through experimentation on several medical imaging datasets, we show that the predictions of the network can be refined using several rounds of training with the prediction from the same weakly annotated data. Further improvements are shown using the clicked points within a custom-designed loss and attention mechanism. Our approach has the potential to speed up the process of generating new training datasets for the development of new machine-learning and deep-learning-based models for, but not exclusively, medical image analysis.

Inter Extreme Points Geodesics for End-to-End Weakly Supervised Image Segmentation

“…at inference stage [6,7,14,24,15,2,9]. Bounding boxes are the most commonly used weak annotations [6,7,14].…”

“…However, extreme points are more time-efficient while providing extra information [12]. To our knowledge, only one extreme points supervision technique has been proposed for 3D medical image segmentation [15]. This method alternates between pseudo-mask generation using Random Walker and network training to mimic these masks.…”

“…Weakly supervised image segmentation Weakly supervised learning covers a large variety of techniques that aim to build predictive models using time-efficient weak annotations. Most existing weakly supervised approaches [14,24,15] adopt an iterative training strategy that alternates between pseudomask (proposal) generations using non-learning techniques (e.g. Random Walker, CRFs) and training a CNN to mimic these proposals.…”

“…Extreme points as weak annotations Different weak annotations have been proposed such as scribbles [2,9,18,19], bounding boxes [14,7] and extreme points [10,15]. Examples are shown in Figure 1.…”

Arthroscope Localization in 3D Ultrasound Volumes Using Weakly Supervised Deep Learning

“…Therefore, the labelling effort is greatly reduced. Weakly supervised deep learning approaches have been used before to detect anatomical structures on US images of the placenta [26,27], breast [28,29], lung [30,31] and the brain [32,33]. These approaches use a combination of (strongly) supervised and weakly supervised methods.…”