MedRegNet: unsupervised multimodal retinal-image registration with GANs and ranking loss

Chronological Registration of OCT and Autofluorescence Findings in CSCR: Two Distinct Patterns in Disease Course

“…Our segmentation network was trained on additional annotated OCT volumes, for which no corresponding FAF images were available. Our utilized registration network (see Section 2.2.4 ) was trained on additional FAF, fluorescein angiography (FAG) and fundus images [ 8 ]. The subsets of our data as used for training and different evaluations are given in Table 1 .…”

“…Our visualisation pipeline itself consists of the components: (1) the mapping function for projecting segmentations from OCT B-scans onto the corresponding IR image, (2) the shape filling algorithm to interpolate segmentations in the gaps between the B-scans, and (3) the MedRegNet registration module [ 8 ] to register the IR image and its segmentations onto the FAF image. Each component including the segmentation network is explained in the following subsections.…”

“…To project the interpolated labels onto the FAF image, we need to register the IR image with the FAF image. For this, we used the feature-based multimodal MedRegNet descriptor module [ 8 ], as depicted in Figure 4 . MedRegNet utilizes established point-detector algorithms such as SIFT [ 37 ] to detect interest points in both images.…”

“…For our usecase, we utilized the multi- and monomodally pre-trained MedRegNet models of [ 8 ] without further training. We referred to the latter as MedRegNet .…”

“…Previous works have proposed several methods for the registration of FAF and other en face imaging modalities [ 8 , 9 , 10 , 11 , 12 , 13 , 14 , 15 ]. Similarly, efforts have been made to visualize OCT labels, e.g., by projecting them onto the accompanying en face infrared (IR) image, where B-scan positions are marked by the OCT device [ 16 , 17 ] or a joint reference space [ 18 ].…”