Deep multi-instance heatmap regression for the detection of retinal vessel crossings and bifurcations in eye fundus images

Hervella, Álvaro S.; Rouco, José; Novo, Jorge; Penedo, Manuel G.; Ortega, Marcos

doi:10.1016/j.cmpb.2019.105201

Cited by 35 publications

(22 citation statements)

References 28 publications

(58 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Our previous work on the same data base [24] formulates the landmark detection task as a deep learning-based approach, and demonstrates first results on intra-operative and surgical simulator datasets for heart surgeries. Hervella et al [15] demonstrated a similar method for the case of retinal fundus images. However, the model has to learn from a heavily unbalanced dataset due to the nature of point landmarks in the context of a segmentation task.…”

Section: Heatmap-based Approachesmentioning

confidence: 99%

Point detection through multi-instance deep heatmap regression for sutures in endoscopy

et al. 2021

View full text Add to dashboard Cite

Purpose: Mitral valve repair is a complex minimally invasive surgery of the heart valve. In this context, suture detection from endoscopic images is a highly relevant task that provides quantitative information to analyse suturing patterns, assess prosthetic configurations and produce augmented reality visualisations. Facial or anatomical landmark detection tasks typically contain a fixed number of landmarks, and use regression or fixed heatmap-based approaches to localize the landmarks. However in endoscopy, there are a varying number of sutures in every image, and the sutures may occur at any location in the annulus, as they are not semantically unique. Method: In this work, we formulate the suture detection task as a multi-instance deep heatmap regression problem, to identify entry and exit points of sutures. We extend our previous work, and introduce the novel use of a 2D Gaussian layer followed by a differentiable 2D spatial Soft-Argmax layer to function as a local non-maximum suppression. Results: We present extensive experiments with multiple heatmap distribution functions and two variants of the proposed model. In the intra-operative domain, Variant 1 showed a mean $$F_1$$ F 1 of $$+ 0.0422$$ + 0.0422 over the baseline. Similarly, in the simulator domain, Variant 1 showed a mean $$F_1$$ F 1 of $$+ 0.0865$$ + 0.0865 over the baseline. Conclusion: The proposed model shows an improvement over the baseline in the intra-operative and the simulator domains. The data is made publicly available within the scope of the MICCAI AdaptOR2021 Challenge https://adaptor2021.github.io/, and the code at https://github.com/Cardio-AI/suture-detection-pytorch/.

show abstract

Section: Heatmap-based Approachesmentioning

confidence: 99%

Point detection through multi-instance deep heatmap regression for sutures in endoscopy

et al. 2021

View full text Add to dashboard Cite

show abstract

“…For imagery, AI healthcare systems may use graphs to show the relative probability of different outcomes or the relative importance of different symptoms for those outcomes, which is more akin to how the LIME algorithm [58] works for diagnostic features. Physicians may present visualization differently from how AI systems offer visual explanations, but even the use of x-rays and other test reports are generally accompanied by explanations highlighting the location of critical signs indicating a diagnosis-with a similar goal as gradient-based heatmaps [59][60][61][62] in XAI systems. Of course, the particular visualizations provided by algorithms such as LIME [63] may themselves be hard to understand.…”

Section: Methods For Providing Explanationsmentioning

confidence: 99%

Examining the effect of explanation on satisfaction and trust in AI diagnostic systems

Alam

2021

BMC Med Inform Decis Mak

View full text Add to dashboard Cite

Background Artificial Intelligence has the potential to revolutionize healthcare, and it is increasingly being deployed to support and assist medical diagnosis. One potential application of AI is as the first point of contact for patients, replacing initial diagnoses prior to sending a patient to a specialist, allowing health care professionals to focus on more challenging and critical aspects of treatment. But for AI systems to succeed in this role, it will not be enough for them to merely provide accurate diagnoses and predictions. In addition, it will need to provide explanations (both to physicians and patients) about why the diagnoses are made. Without this, accurate and correct diagnoses and treatments might otherwise be ignored or rejected. Method It is important to evaluate the effectiveness of these explanations and understand the relative effectiveness of different kinds of explanations. In this paper, we examine this problem across two simulation experiments. For the first experiment, we tested a re-diagnosis scenario to understand the effect of local and global explanations. In a second simulation experiment, we implemented different forms of explanation in a similar diagnosis scenario. Results Results show that explanation helps improve satisfaction measures during the critical re-diagnosis period but had little effect before re-diagnosis (when initial treatment was taking place) or after (when an alternate diagnosis resolved the case successfully). Furthermore, initial “global” explanations about the process had no impact on immediate satisfaction but improved later judgments of understanding about the AI. Results of the second experiment show that visual and example-based explanations integrated with rationales had a significantly better impact on patient satisfaction and trust than no explanations, or with text-based rationales alone. As in Experiment 1, these explanations had their effect primarily on immediate measures of satisfaction during the re-diagnosis crisis, with little advantage prior to re-diagnosis or once the diagnosis was successfully resolved. Conclusion These two studies help us to draw several conclusions about how patient-facing explanatory diagnostic systems may succeed or fail. Based on these studies and the review of the literature, we will provide some design recommendations for the explanations offered for AI systems in the healthcare domain.

show abstract

“…Pattern recognition and machine learning approaches can be used for differentiating between crossings and bifurcations. 18 , 27 – 29 But this is still an active area of research. Therefore, detecting spurious bifurcation points in 2D samples should be expected.…”

Section: Image Analysis and Quantificationsmentioning

confidence: 99%

Unbiased analysis of mouse brain endothelial networks from two- or three-dimensional fluorescence images

et al. 2022

View full text Add to dashboard Cite

. Significance: A growing body of research supports the significant role of cerebrovascular abnormalities in neurological disorders. As these insights develop, standardized tools for unbiased and high-throughput quantification of cerebrovascular structure are needed. Aim: We provide a detailed protocol for performing immunofluorescent labeling of mouse brain vessels, using thin ( ) or thick (50 to ) tissue sections, followed respectively by two- or three-dimensional (2D or 3D) unbiased quantification of vessel density, branching, and tortuosity using digital image processing algorithms. Approach: Mouse brain sections were immunofluorescently labeled using a highly selective antibody raised against mouse Cluster of Differentiation-31 (CD31), and 2D or 3D microscopy images of the mouse brain vasculature were obtained using optical sectioning. An open-source toolbox, called Pyvane, was developed for analyzing the imaged vascular networks. The toolbox can be used to identify the vasculature, generate the medial axes of blood vessels, represent the vascular network as a graph, and calculate relevant measurements regarding vascular morphology. Results: Using Pyvane, vascular parameters such as endothelial network density, number of branching points, and tortuosity are quantified from 2D and 3D immunofluorescence micrographs. Conclusions: The steps described in this protocol are simple to follow and allow for reproducible and unbiased analysis of mouse brain vascular structure. Such a procedure can be applied to the broader field of vascular biology.

show abstract

Deep multi-instance heatmap regression for the detection of retinal vessel crossings and bifurcations in eye fundus images

Cited by 35 publications

References 28 publications

Point detection through multi-instance deep heatmap regression for sutures in endoscopy

Point detection through multi-instance deep heatmap regression for sutures in endoscopy

Examining the effect of explanation on satisfaction and trust in AI diagnostic systems

Unbiased analysis of mouse brain endothelial networks from two- or three-dimensional fluorescence images

Contact Info

Product

Resources

About