Deep Learning Based Multi-modal Registration for Retinal Imaging

Arikan, Mustafa; Sadeghipour, Amir; Gerendas, Bianca S.; Told, Reinhard; Schmidt-Erfurt, Ursula

doi:10.1007/978-3-030-33850-3_9

Cited by 11 publications

(12 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…They applied their method on different combinations of modalities including fundus autofluorescence (FAF) to scanning laser ophthalmoscopy (SLO)/OCT, fluorescein angiogram (FA) to OCT‐angiography (OCT‐A), and indocyanine green angiography (ICGA) to FA and compared their algorithm with intensity based multimodal affine registrations from elastix. They achieved an average error rate of 13.12% versus 30.8% using the reference method, which confirmed that multi‐modal registration of retinal images using vessel segmentation and landmark detection was more accurate than reference methods 12 . Lee et al introduced a non‐human dependent method for multimodal retinal image registration that is based on CNN 13 .…”

Section: Discussionmentioning

confidence: 80%

“…Arikan et al used vessel segmentation and automatic landmark detection to establish image registration, the process of aligning two or more images based on image appearances. They developed an algorithm allowing for tracing changes in retinal structure across different retinal imaging modalities 12 . They performed automatic multi‐modal retinal 2D/3D image registration using U‐Net for vessel segmentation and mask regional CNNs (R‐CNN) for detection of vessel landmarks (bifurcations, branches and crossover).…”

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

Efficacy and accuracy of artificial intelligence to overlay multimodal images from different optical instruments in patients with retinitis pigmentosa

Yassin

Wang

Freeman

et al. 2023

Clinical Exper Ophthalmology

View full text Add to dashboard Cite

Background: Retinitis pigmentosa (RP) represents a group of progressive, genetically heterogenous blinding diseases. Recently, relationships between measures of retinal function and structure are needed to help identify outcome measures or biomarkers for clinical trials. The ability to align retinal multimodal images, taken on different platforms, will allow better understanding of this relationship. We investigate the efficacy of artificial intelligence (AI) in overlaying different multimodal retinal images in RP patients. Methods:We overlayed infrared images from microperimetry on near-infrared images from scanning laser ophthalmoscope and spectral domain optical coherence tomography in RP patients using manual alignment and AI. The AI adopted a two-step framework and was trained on a separate dataset. Manual alignment was performed using in-house software that allowed labelling of six key points located at vessel bifurcations. Manual overlay was considered successful if the distance between same key points on the overlayed images was ≤1/2 . Results: Fifty-seven eyes of 32 patients were included in the analysis. AI was significantly more accurate and successful in aligning images compared to manual alignment as confirmed by linear mixed-effects modelling (p < 0.001). A receiver operating characteristic analysis, used to compute the area under the curve of the AI (0.991) and manual (0.835) Dice coefficients in relation to their respective 'truth' values, found AI significantly more accurate in the overlay (p < 0.001).Conclusion: AI was significantly more accurate than manual alignment in overlaying multimodal retinal imaging in RP patients and showed the potential to use AI algorithms for future multimodal clinical and research applications.

show abstract

Section: Discussionmentioning

confidence: 80%

Section: Discussionmentioning

confidence: 99%

Efficacy and accuracy of artificial intelligence to overlay multimodal images from different optical instruments in patients with retinitis pigmentosa

Yassin

Wang

Freeman

et al. 2023

Clinical Exper Ophthalmology

View full text Add to dashboard Cite

show abstract

“…More specifically, considering the case of deep learningbased multimodal retinal image registration, there are works where rigid or deformable transformations are obtained using supervised, weakly supervised, or unsupervised approaches [17][18][19][20][21][22][23]. While deformable methods [19,22,23] are more competitive than rigid ones [17,18,20,21], the former generally require that the input image pair is already approximately registered (usually via an affine transformation). To do the latter, there are two options: incorporate this stage into the methodology itself or assume that this step was previously performed with an external method.…”

Section: Related Workmentioning

confidence: 99%

“…In relation to the unsupervised methods [18,19,23], they have the advantage of not requiring segmentation knowledge, but in the context of multimodal images, the absence of this type of knowledge tends to reduce the accuracy of the obtained registration. As for the purely supervised methods [17,20], the requirement of a highquality ground-truth can hinder their applicability, especially in a context associated with medical images and their daily use in clinical practice. Finally, regarding the weakly supervised methods [21,22], such as the one proposed here, they allow us to solve the shortcomings of the previous two (supervised and unsupervised), given that the segmentation knowledge required may be partial and imprecise (it may even contain noise), but it is still useful enough to guide and improve the registration process.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Deformable registration of multimodal retinal images using a weakly supervised deep learning approach

Martínez-Río

Carmona

Cancelas

et al. 2023

Neural Comput & Applic

View full text Add to dashboard Cite

There are different retinal vascular imaging modalities widely used in clinical practice to diagnose different retinal pathologies. The joint analysis of these multimodal images is of increasing interest since each of them provides common and complementary visual information. However, if we want to facilitate the comparison of two images, obtained with different techniques and containing the same retinal region of interest, it will be necessary to make a previous registration of both images. Here, we present a weakly supervised deep learning methodology for robust deformable registration of multimodal retinal images, which is applied to implement a method for the registration of fluorescein angiography (FA) and optical coherence tomography angiography (OCTA) images. This methodology is strongly inspired by VoxelMorph, a general unsupervised deep learning framework of the state of the art for deformable registration of unimodal medical images. The method was evaluated in a public dataset with 172 pairs of FA and superficial plexus OCTA images. The degree of alignment of the common information (blood vessels) and preservation of the non-common information (image background) in the transformed image were measured using the Dice coefficient (DC) and zero-normalized cross-correlation (ZNCC), respectively. The average values of the mentioned metrics, including the standard deviations, were DC = 0.72 ± 0.10 and ZNCC = 0.82 ± 0.04. The time required to obtain each pair of registered images was 0.12 s. These results outperform rigid and deformable registration methods with which our method was compared.

show abstract

Developing a Three Dimensional Registration Method for Optical Coherence Tomography Data

Vadgama

Logofătu

Thoma

2021

Advances in Computational Collective Intelligence

View full text Add to dashboard Cite

Deep Learning Based Multi-modal Registration for Retinal Imaging

Cited by 11 publications

References 13 publications

Efficacy and accuracy of artificial intelligence to overlay multimodal images from different optical instruments in patients with retinitis pigmentosa

Efficacy and accuracy of artificial intelligence to overlay multimodal images from different optical instruments in patients with retinitis pigmentosa

Deformable registration of multimodal retinal images using a weakly supervised deep learning approach

Developing a Three Dimensional Registration Method for Optical Coherence Tomography Data

Contact Info

Product

Resources

About