A Semantic Fusion Approach Between Medical Images and Reports Using UMLS

Racoceanu, Daniel; Lacoste, Caroline; Teodorescu, Roxana; Vuillemenot, Nicolas

doi:10.1007/11880592_35

Cited by 11 publications

(9 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…[324] R. Teodorescu, C. Cernazanu-Glavan, V. Cretu, D. Racoceanu, The use of the medical ontology for a semantic-based fusion system in biomedical informatics application to Alzheimer's disease, in: [19], tissue classification [20], brain diagnosis [20], classifier fusion [21], breast cancer tumor detection [21,22], delineation & recognition of anatomical brain object [18] and medical image retrieval [23,24,25] [36], classification [36], fusion [36,19,27,37,38,27,39,40,41,42,43], micro-calcification diagnosis [19], breast cancer detection [38,44,45], medical diagnosis [27,28,42] [47,48,49,50], cancer treatment [51], image segmentation and integration [51,52], maximization mutual information [53], deep brain stimulation [54], brain tumor segmentation [55], image retrieval [56,57], spatial weighted entropy [56], feature fusion…”

Section: Discussionmentioning

confidence: 99%

Medical image fusion: A survey of the state of the art

2014

View full text Add to dashboard Cite

Medical image fusion is the process of registering and combining multiple images from single or multiple imaging modalities to improve the imaging quality and reduce randomness and redundancy in order to increase the clinical applicability of medical images for diagnosis and assessment of medical problems. Multi-modal medical image fusion algorithms and devices have shown notable achievements in improving clinical accuracy of decisions based on medical images. This review article provides a factual listing of methods and summarizes the broad scientific challenges faced in the field of medical image fusion. We characterize the medical image fusion research based on (1) the widely used image fusion methods, (2) imaging modalities, and (3) imaging of organs that are under study. This review concludes that even though there exists several open ended technological and scientific challenges, the fusion of medical images has proved to be useful for advancing the clinical reliability of using medical imaging for medical diagnostics and analysis, and is a scientific discipline that has the potential to significantly grow in the coming years.

show abstract

Section: Discussionmentioning

confidence: 99%

Medical image fusion: A survey of the state of the art

2014

View full text Add to dashboard Cite

show abstract

“…For example, in [29,30], the characteristics of intensity, hue and saturation are used for this. The selected features can be semantically labelled using anatomical brain atlases [31], special visual indexes [31] or ontologies [32]. The ontology describes the medical terms via a controlled vocabulary, where the conceptualizations of the domain knowledge are constructed as an OWL (Ontology Web Language) model.…”

Section: Background and Related Workmentioning

confidence: 99%

Comparative assessment of text-image fusion models for medical diagnostics

Gusarova

Lobantsev

Vatian

et al. 2020

ICS

View full text Add to dashboard Cite

Introduction: Information overload and complexity are characteristic of decision-making in medicine. In these conditions information fusion techniques are effective. For the diagnosis and treatment of pneumonia using x-ray images and accompaniyng free-text radiologists reports, it is promising to use text-image fusion. Purpose: Development of a method for fusing text with an image in the treatment and diagnosis of pneumonia using neural networks. Methods: We used MIMIC-CXR dataset, the SEResNeXt101-32x4d for images feature extracting and the Bio-ClinicalBERT model followed by ContextLSTM layer for text featureextracting. We compared five architectures in the conducted experiment: image classifier, report classifier and three scenarios of the fusion, namely late fusion, middle fusion and early fusion. Results: We got an absolute excess of metrics (ROC AUC = 0.9933, PR AUC = 0.9907) when using an early fusion classifier (ROC AUC = 0.9921, PR AUC = 0.9889) even over the idealized case of text classifier (that is, without taking into account possible errors of the radiologist). The network training time ranged from 20 minutes for late fusion to 9 hours and 45 minutes for early fusion. Based on Class Activation Map technique we graphically showed that the image feature extractor in the fused classification scenario still learns discriminative regions for pneumonia classification problem. Discussion: Fusing text and images increases the likelihood of correct image classification compared to only image classification. The proposed combined image-report classifier trained with the early-fusion method gives better performance than individual classifiers in the pneumonia classification problem. However, it is worth considering that better results cost the training time and required computation resources. Report-based training is much faster in training and less demanding for computation capacity.

show abstract

“…In medical informatics, triple-based representation formats have been popularized since the first release of the UMLS metathesaurus [127], but they reach back to [128]. Occasionally, even UMLS has been described as an ontology [129,130], but our impression is that -at least in biomedical informatics literature -this terminological confusion has gradually been overcome. It had been a merit of the Semantic Web to The usefulness of formalized upper-level ontologies (Fig.…”

Section: Recommendationsmentioning

confidence: 99%

Formal Ontologies in Biomedical Knowledge Representation

Schulz

Jansen²

2013

Yearb Med Inform

View full text Add to dashboard Cite

SummaryObjectives: Medical decision support and other intelligent applications in the life sciences depend on increasing amounts of digital information. Knowledge bases as well as formal ontologies are being used to organize biomedical knowledge and data. However, these two kinds of artefacts are not always clearly distinguished. Whereas the popular RDF(S) standard provides an intuitive triple-based representation, it is semantically weak. Description logics based ontology languages like OWL-DL carry a clear-cut semantics, but they are computationally expensive, and they are often misinterpreted to encode all kinds of statements, including those which are not ontological. Method: We distinguish four kinds of statements needed to comprehensively represent domain knowledge: universal statements, terminological statements, statements about particulars and contingent statements. We argue that the task of formal ontologies is solely to represent universal statements, while the non-ontological kinds of statements can nevertheless be connected with ontological representations. To illustrate these four types of representations, we use a running example from parasitology. Results: We finally formulate recommendations for semantically adequate ontologies that can efficiently be used as a stable framework for more context-dependent biomedical knowledge representation and reasoning applications like clinical decision support systems.

show abstract

A Semantic Fusion Approach Between Medical Images and Reports Using UMLS

Cited by 11 publications

References 23 publications

Medical image fusion: A survey of the state of the art

Medical image fusion: A survey of the state of the art

Comparative assessment of text-image fusion models for medical diagnostics

Formal Ontologies in Biomedical Knowledge Representation

Contact Info

Product

Resources

About