This contribution presents a deep learning method for the extraction and fusion of information relating to kidney stone fragments acquired from different viewpoints of the endoscope. Surface and section fragment images are jointly used during the training of the classifier to improve the discrimination power of the features by adding attention layers at the end of each convolutional block. This approach is specifically designed to mimic the morpho-constitutional analysis performed in ex-vivo by biologists to visually identify kidney stones by inspecting both views. The addition of attention mechanisms to the backbone improved the results of single view extraction backbones by 4% on average. Moreover, in comparison to the state-of-the-art, the fusion of the deep features improved the overall results up to 11% in terms of kidney stone classification accuracy.
Cette contribution présente une méthode d'apprentissage profond pour l'extraction et la fusion d'informations d'images acquises sous différents points de vue dans le but de produire des caractéristiques plus discriminantes entre objets. Notre approche a été conc ¸ue pour mimer l'analyse morpho-constitutionnelle utilisée par les urologues pour classer visuellement des fragments de calculs rénaux à partir de leur surface et section. Des stratégies de fusion de caractéristiques profondes ont permis d'améliorer les performances des extracteurs (structures principales des réseaux) à vue unique de plus de 10 % en termes de précision de la classification des calculs rénaux.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.