Two-Step Registration on Multi-Modal Retinal Images via Deep Neural Networks

Zhang, Junkang; Wang, Yiqian; Dai, Ji; Cavichini, Melina; Bartsch, Dirk-Uwe; Freeman, William R.; Nguyen, Truong Q.; An, Cheolhong

doi:10.1109/tip.2021.3135708

Cited by 23 publications

(32 citation statements)

References 66 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…More specifically, considering the case of deep learningbased multimodal retinal image registration, there are works where rigid or deformable transformations are obtained using supervised, weakly supervised, or unsupervised approaches [17][18][19][20][21][22][23]. While deformable methods [19,22,23] are more competitive than rigid ones [17,18,20,21], the former generally require that the input image pair is already approximately registered (usually via an affine transformation). To do the latter, there are two options: incorporate this stage into the methodology itself or assume that this step was previously performed with an external method.…”

Section: Related Workmentioning

confidence: 99%

“…In relation to the unsupervised methods [18,19,23], they have the advantage of not requiring segmentation knowledge, but in the context of multimodal images, the absence of this type of knowledge tends to reduce the accuracy of the obtained registration. As for the purely supervised methods [17,20], the requirement of a highquality ground-truth can hinder their applicability, especially in a context associated with medical images and their daily use in clinical practice.…”

Section: Related Workmentioning

confidence: 99%

“…On the other hand, a two-step unsupervised learning framework based on deep convolutional networks is proposed in [23], which is used to register CFP and FA images. In the first step, three sequentially connected networks are used to carry out a coarse alignment based on vessel segmentation information.…”

Section: Related Workmentioning

confidence: 99%

See 2 more Smart Citations

Deformable registration of multimodal retinal images using a weakly supervised deep learning approach

Martínez-Río

Carmona

Cancelas

et al. 2023

Neural Comput & Applic

View full text Add to dashboard Cite

There are different retinal vascular imaging modalities widely used in clinical practice to diagnose different retinal pathologies. The joint analysis of these multimodal images is of increasing interest since each of them provides common and complementary visual information. However, if we want to facilitate the comparison of two images, obtained with different techniques and containing the same retinal region of interest, it will be necessary to make a previous registration of both images. Here, we present a weakly supervised deep learning methodology for robust deformable registration of multimodal retinal images, which is applied to implement a method for the registration of fluorescein angiography (FA) and optical coherence tomography angiography (OCTA) images. This methodology is strongly inspired by VoxelMorph, a general unsupervised deep learning framework of the state of the art for deformable registration of unimodal medical images. The method was evaluated in a public dataset with 172 pairs of FA and superficial plexus OCTA images. The degree of alignment of the common information (blood vessels) and preservation of the non-common information (image background) in the transformed image were measured using the Dice coefficient (DC) and zero-normalized cross-correlation (ZNCC), respectively. The average values of the mentioned metrics, including the standard deviations, were DC = 0.72 ± 0.10 and ZNCC = 0.82 ± 0.04. The time required to obtain each pair of registered images was 0.12 s. These results outperform rigid and deformable registration methods with which our method was compared.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Deformable registration of multimodal retinal images using a weakly supervised deep learning approach

Martínez-Río

Carmona

Cancelas

et al. 2023

Neural Comput & Applic

View full text Add to dashboard Cite

show abstract

“…Different researchers provided various solutions. Some researchers adopted deep learning in the feature detection process and further aligned the feature points using conventional image alignment methods, such as RANSAC [40,41], while some work constructed an outlier-rejection network to compute the image transformation matrix [45,46]. There is no consensus on how deep learning should be added to the registration pipeline [46].…”

Section: Figurementioning

confidence: 99%

“…Some researchers adopted deep learning in the feature detection process and further aligned the feature points using conventional image alignment methods, such as RANSAC [40,41], while some work constructed an outlier-rejection network to compute the image transformation matrix [45,46]. There is no consensus on how deep learning should be added to the registration pipeline [46]. Moreover, in most deep learningbased registration algorithms, accurate registration relies on accurate segmentation, which is still an ongoing research topic in medical image processing.…”

Section: Figurementioning

confidence: 99%

Color fundus photograph registration based on feature and intensity for longitudinal evaluation of diabetic retinopathy progression

Zhou

Jin

et al. 2022

Front. Phys.

View full text Add to dashboard Cite

Longitudinal disease progression evaluation between follow-up examinations relies on precise registration of medical images. Compared to other medical imaging methods, color fundus photograph, a common retinal examination, is easily affected by eye movements while shooting, for which we think it is necessary to develop a reliable longitudinal registration method for this modality. Thus, the purpose of this study was to propose a robust registration method for longitudinal color fundus photographs and establish a longitudinal retinal registration dataset. In the proposed algorithm, radiation-variation insensitive feature transform (RIFT) feature points were calculated and aligned, followed by further refinement using a normalized total gradient (NTG). Experiments and ablation analyses were conducted on both public and private datasets, using the mean registration error and registration success plot as the main evaluation metrics. The results showed that our proposed method was comparable to other state-of-the-art registration algorithms and was particularly accurate for longitudinal images with disease progression. We believe the proposed method will be beneficial for the longitudinal evaluation of fundus images.

show abstract

Retinal vessel segmentation to diagnose diabetic retinopathy using fundus images: A survey

Radha

Karuna

2023

Int J Imaging Syst Tech

View full text Add to dashboard Cite

Diabetes can cause damage to the retina's blood vessels in the eye leading to diabetic retinopathy (DR). The images captured using a fundus camera are used to segment and study the blood vessel damage. Once the retina is damaged, it cannot be repaired. Therefore, early detection of blood vessel damage is the only way to control the progression of the disease. It allows physicians to provide timely and appropriate treatment to patients. The ophthalmologist can manually recognize and mark these vessels based on some clinical and geometrical features; however, it is time‐consuming. Extraction and segmentation play a significant role in showing the difference between healthy and newly developed abnormal vessels. Due to the increased diabetes population, automated systems have been designed to detect retinal blood vessels and assist ophthalmologists. Identifying, extracting, and examining blood vessels are intricate processes, specifically identifying new vessels in the retina. In medical image analysis, artificial intelligence and deep learning techniques have become widely used practices for automatic retinal blood vessel segmentation. We reviewed articles from 1989 to 2023, including handcrafted segmentation to recent deep‐learning techniques with available public datasets. We have concluded this article with an overview of observed parameters, calculations, and future directions for the analysis of retinal images. We believe this review article will help researchers identify research gaps in the field of DR.

show abstract

Two-Step Registration on Multi-Modal Retinal Images via Deep Neural Networks

Cited by 23 publications

References 66 publications

Deformable registration of multimodal retinal images using a weakly supervised deep learning approach

Deformable registration of multimodal retinal images using a weakly supervised deep learning approach

Color fundus photograph registration based on feature and intensity for longitudinal evaluation of diabetic retinopathy progression

Retinal vessel segmentation to diagnose diabetic retinopathy using fundus images: A survey

Contact Info

Product

Resources

About