The 2021 ImageCLEF Benchmark: Multimedia Retrieval in Medical, Nature, Internet and Social Media Applications

Ionescu, Bogdan; Müller, Henning; Péteri, Renaud; Abacha, Asma Ben; Demner‐Fushman, Dina; Hasan, Sadid A.; Sarrouti, Mourad; Pelka, Obioma; Friedrich, Christoph M.; Herrera, Alba García Seco de; Jacutprakart, Janadhip; Kovalev, Vassili; Kozlovski, Serge; Liauchuk, Vitali; Cid, Yashin Dicente; Chamberlain, Jon; Clark, Adrian; Campello, Antonio; Moustahfid, Hassan; Oliver, Thomas; Schulz, Abigail; Brie, Paul; Berari, Raul; Fichou, Dimitri; Tauteanu, Andrei; Dogariu, Mihai; Ştefan, Liviu-Daniel; Constantin, Mihai Gabriel; Deshayes, Jérôme; Popescu, Adrian

doi:10.1007/978-3-030-72240-1_72

Cited by 3 publications

(14 citation statements)

References 13 publications

(10 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We build on the ImageCLEF 2021 Aware shared evaluation task [20], which proposes a dataset for raising user awareness. A set of 500 photographic profiles sampled from the YFCC dataset [46], itself collected from Flickr, were manually rated in 4 real-life situations.…”

Section: Related Workmentioning

confidence: 99%

“…We motivate it and present its components. Users share their [20] personal data on OSNs in an initial context which they choose and control. These data are then aggregated into profiles which can be used in secondary contexts which are were initially anticipated.…”

Section: Proposed Contribution 31 Motivationmentioning

confidence: 99%

“…Users should be able to understand the potential effects of their data-sharing practices. This topic was explored during the Image-CLEF 2021 Aware Task [20] which introduced a dataset which includes photographic user profiles, associating object detections and human ratings of these profiles in four real life situations. The modeled situations include search for: an accommodation, a bank loan, an IT job and a waiter job.…”

Section: Introductionmentioning

confidence: 99%

“…Inversely, 𝑈 2 will be low ranked for waiter job search because their photos are either unrelated to this situation or can be judged negatively. Note that the ratings of the profiles encode social biases of the people who perform the ratings [20,31]. The automatic rating process will inherit these biases, but this is not considered problematic in our approach, because it is meant to simulate real-life situations, including biases.…”

Section: Introductionmentioning

confidence: 99%

“…We explore different ways to structure data as graphs and three GNN models to adapt the automatic rating of profiles to each situation. We evaluate the proposed model on the publicly available ImageCLEF Aware dataset [20], which contains a total of 500 profiles. Results show a consistent improvement compared to a newly proposed baseline and to an existing method.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Raising User Awareness about the Consequences of Online Photo Sharing

Schindler

Popescu

Nguyen

et al. 2023

Proceedings of the 2023 ACM International Conference on Multimedia Retrieval

View full text Add to dashboard Cite

Online social networks use AI techniques to automatically infer profiles from users' shared data. However, these inferences and their effects remain, to a large extent, opaque to the users themselves. We propose a method which raises user awareness about the potential use of their profiles in impactful situations, such as searching for a job or an accommodation. These situations illustrate usage contexts that users might not have anticipated when deciding to share their data. User photographic profiles are described by automatic object detections in profile photos, and associated object ratings in situations. Human ratings of the profiles per situation are also available for training. These data are represented as graph structures which are fed into graph neural networks in order to learn how to automatically rate them. An adaptation of the learning procedure per situation is proposed since the same profile is likely to be interpreted differently, depending on the context. Automatic profile ratings are compared to one another in order to inform individual users of their standing with respect to others. Our method is evaluated on a public dataset, and consistently outperforms competitive baselines. An ablation study gives insights about the role of its main components.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Proposed Contribution 31 Motivationmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Raising User Awareness about the Consequences of Online Photo Sharing

Schindler

Popescu

Nguyen

et al. 2023

Proceedings of the 2023 ACM International Conference on Multimedia Retrieval

View full text Add to dashboard Cite

show abstract

AI co-pilot: content-based image retrieval for the reading of rare diseases in chest CT

Haubold

Zeng²,

Farhand³

et al. 2023

Sci Rep

View full text Add to dashboard Cite

The aim of the study was to evaluate the impact of the newly developed Similar patient search (SPS) Web Service, which supports reading complex lung diseases in computed tomography (CT), on the diagnostic accuracy of residents. SPS is an image-based search engine for pre-diagnosed cases along with related clinical reference content (https://eref.thieme.de). The reference database was constructed using 13,658 annotated regions of interest (ROIs) from 621 patients, comprising 69 lung diseases. For validation, 50 CT scans were evaluated by five radiology residents without SPS, and three months later with SPS. The residents could give a maximum of three diagnoses per case. A maximum of 3 points was achieved if the correct diagnosis without any additional diagnoses was provided. The residents achieved an average score of 17.6 ± 5.0 points without SPS. By using SPS, the residents increased their score by 81.8% to 32.0 ± 9.5 points. The improvement of the score per case was highly significant (p = 0.0001). The residents required an average of 205.9 ± 350.6 s per case (21.9% increase) when SPS was used. However, in the second half of the cases, after the residents became more familiar with SPS, this increase dropped to 7%. Residents’ average score in reading complex chest CT scans improved by 81.8% when the AI-driven SPS with integrated clinical reference content was used. The increase in time per case due to the use of the SPS was minimal.

show abstract

Diversity Learning Based on Multi-Latent Space for Medical Image Visual Question Generation

Zhu

Togo

Ogawa

et al. 2023

Sensors

View full text Add to dashboard Cite

Auxiliary clinical diagnosis has been researched to solve unevenly and insufficiently distributed clinical resources. However, auxiliary diagnosis is still dominated by human physicians, and how to make intelligent systems more involved in the diagnosis process is gradually becoming a concern. An interactive automated clinical diagnosis with a question-answering system and a question generation system can capture a patient’s conditions from multiple perspectives with less physician involvement by asking different questions to drive and guide the diagnosis. This clinical diagnosis process requires diverse information to evaluate a patient from different perspectives to obtain an accurate diagnosis. Recently proposed medical question generation systems have not considered diversity. Thus, we propose a diversity learning-based visual question generation model using a multi-latent space to generate informative question sets from medical images. The proposed method generates various questions by embedding visual and language information in different latent spaces, whose diversity is trained by our newly proposed loss. We have also added control over the categories of generated questions, making the generated questions directional. Furthermore, we use a new metric named similarity to accurately evaluate the proposed model’s performance. The experimental results on the Slake and VQA-RAD datasets demonstrate that the proposed method can generate questions with diverse information. Our model works with an answering model for interactive automated clinical diagnosis and generates datasets to replace the process of annotation that incurs huge labor costs.

show abstract

The 2021 ImageCLEF Benchmark: Multimedia Retrieval in Medical, Nature, Internet and Social Media Applications

Cited by 3 publications

References 13 publications

Raising User Awareness about the Consequences of Online Photo Sharing

Raising User Awareness about the Consequences of Online Photo Sharing

AI co-pilot: content-based image retrieval for the reading of rare diseases in chest CT

Diversity Learning Based on Multi-Latent Space for Medical Image Visual Question Generation

Contact Info

Product

Resources

About