Automatic Image Annotation Combining the Content and the Context of Medical Images

Florea, Filip; Buzuloiu, Vasile; Rogozan, Alexandrina; Bensrhair, Abdelaziz; Darmoni, Stéfan Jacques

doi:10.1109/isscs.2007.4292712

Cited by 3 publications

(3 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The combination of methods relating to image content and context has been identified as a possible means to bridging the semantic gap in multiple domains (Florea, Buzuloiu, Rogozan, Bensrhair, & Darmoni, ; Tuffield et al., ; Smeulders et al., ). One particular type of context, which multiple authors have identified as relevant, is location.…”

Section: Related Work On Image Indexing and Retrievalmentioning

confidence: 99%

User evaluation of automatically generated keywords and toponyms for geo‐referenced images

Ostermann¹,

Tomko

Purves

2013

J Am Soc Inf Sci Tec

View full text Add to dashboard Cite

This article presents the results of a user evaluation of automatically generated concept keywords and place names (toponyms) for geo-referenced images. Automatically annotating images is becoming indispensable for effective information retrieval, since the number of geo-referenced images available online is growing, yet many images are insufficiently tagged or captioned to be efficiently searchable by standard information retrieval procedures. The Tripod project developed original methods for automatically annotating geo-referenced images by generating representations of the likely visible footprint of a geo-referenced image, and using this footprint to query spatial databases and web resources. These queries return raw lists of potential keywords and toponyms, which are subsequently filtered and ranked. This article reports on user experiments designed to evaluate the quality of the generated annotations. The experiments combined quantitative and qualitative approaches: To retrieve a large number of responses, participants rated the annotations in standardized online questionnaires that showed an image and its corresponding keywords. In addition, several focus groups provided rich qualitative information in open discussions. The results of the evaluation show that currently the annotation method performs better on rural images than on urban ones. Further, for each image at least one suitable keyword could be generated. The integration of heterogeneous data sources resulted in some images having a high level of noise in the form of obviously wrong or spurious keywords. The article discusses the evaluation itself and methods to improve the automatic generation of annotations.

show abstract

Section: Related Work On Image Indexing and Retrievalmentioning

confidence: 99%

User evaluation of automatically generated keywords and toponyms for geo‐referenced images

Ostermann¹,

Tomko

Purves

2013

J Am Soc Inf Sci Tec

View full text Add to dashboard Cite

show abstract

“…One solution to this problem is the use of ML tools capable of labeling weighting and sequence type. Related works have proposed ML models that label MR sequences, some by looking directly at the images as matrix data [12]- [14], while others make use of the DICOM metadata [15]- [17]. Solutions that use the DICOM metadata as inputs usually offer very short inference times and require less computational power (no GPU needed) than image-based solutions (that use matrices as input) since metadata inferences are simpler.…”

Section: Introductionmentioning

confidence: 99%

Automatic magnetic resonance imaging series labelling for large repositories

Maya,

Alberich,

Canuto

et al. 2024

Preprint

View full text Add to dashboard Cite

Large medical image repositories present challenges related to unstructured data. A data enrichment process allows the storage of additional information for fast identification of the content and properties of medical imaging studies. The aim of this study is to develop a metadata enrichment pipeline to facilitate the secondary use of medical images in a high-throughput environment. Our aim was to develop a categorization tool for the MR series to generate standardized tags that identify relevant image characteristics such as patient orientation, sequence type, weighting type, or the presence of fat suppression. Three models that make use of machine learning (ML) and DICOM tags are proposed. The dataset for their development consists of 4,666 MR series from cancer patients, labeled by expert radiologists and acquired from different manufacturers, clinical centers, and anatomical regions, covering as much variability as possible with the aim of making the models generalizable to other databases. Moreover, the inference performance of the end system has been evaluated on 25,596 MR series as well as the final model outputs with an external evaluation set of 1,286 MR series. The weighting model achieves very reliable results with a macro f1-score of 0.88 in the validation set. Junk and chemical shift models achieved scores of 0.82 and 0.83respectively. These results open the door to the automatic application of image post-processing and deep learning algorithms after accurate labeling, minimizing human intervention. Furthermore, the proposed solution can infer thousands of DICOM series in less than 1 minute. Thanks to the fast inference times provided by this solution, it fits well in a big data ecosystem, eliminating any performance issues on ingestion in a semi-real-time environment.

show abstract

“…Current research in the biomedical domain (e.g., Florea et al, 2007), has investigated hybrid approaches to image retrieval, combining elements of content-based image retrieval (CBIR) and annotation-based image retrieval (ABIR). ABIR, compared to the image-only approach of CBIR, offers a practical advantage in that queries can be more naturally specified by a human user (Inoue, 2004).…”

Section: Introductionmentioning

confidence: 99%

Using non-lexical features to identify effective indexing terms for biomedical illustrations

Simpson

Demner‐Fushman

Sneiderman

et al. 2009

Proceedings of the 12th Conference of the European Chapter of the Association for Computational Linguistics on - EACL '09

View full text Add to dashboard Cite

Automatic image annotation is an attractive approach for enabling convenient access to images found in a variety of documents. Since image captions and relevant discussions found in the text can be useful for summarizing the content of images, it is also possible that this text can be used to generate salient indexing terms. Unfortunately, this problem is generally domainspecific because indexing terms that are useful in one domain can be ineffective in others. Thus, we present a supervised machine learning approach to image annotation utilizing non-lexical features 1 extracted from image-related text to select useful terms. We apply this approach to several subdomains of the biomedical sciences and show that we are able to reduce the number of ineffective indexing terms.

show abstract

Automatic Image Annotation Combining the Content and the Context of Medical Images

Cited by 3 publications

References 7 publications

User evaluation of automatically generated keywords and toponyms for geo‐referenced images

User evaluation of automatically generated keywords and toponyms for geo‐referenced images

Automatic magnetic resonance imaging series labelling for large repositories

Using non-lexical features to identify effective indexing terms for biomedical illustrations

Contact Info

Product

Resources

About