BioASQ: A Challenge on Large-Scale Biomedical Semantic Indexing and Question Answering

Balikas, Georgios; Krithara, Anastasia; Partalas, Ioannis; Paliouras, George

doi:10.1007/978-3-319-24471-6_3

Cited by 34 publications

(31 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The BioASQ challenge (Balikas et al, 2015) also evaluated the task of automatically summarizing biomedical texts as part of a question answering system. With the aim of facilitating access to biomedical literature, one of the tasks focused on providing exact answers to English questions written by biomedical experts along with a paragraph-sized summary answer.…”

Section: Task Based Evaluationmentioning

confidence: 99%

The challenging task of summary evaluation: an overview

Lloret

Plaza

Aker

2017

Lang Resources & Evaluation

View full text Add to dashboard Cite

Evaluation is crucial in the research and development of automatic summarization applications, in order to determine the appropriateness of a summary based on different criteria, such as the content it contains, and the way it is presented. To perform an adequate evaluation is of great relevance to ensure that automatic summaries can be useful for the context and/or application they are generated for. To this end, researchers must be aware of the evaluation metrics, approaches, and datasets that are available, in order to decide which of them would be the most suitable to use, or to be able to propose new ones, overcoming the possible limitations that existing methods may present. In this article, a critical and historical analysis of evaluation metrics, methods, and datasets for automatic summarization systems is presented, where the strengths and weaknesses of evaluation efforts are discussed and the major challenges to solve are identified. Therefore, a clear up-to-date overview of the evolution and progress of summarization evaluation is provided, giving the reader useful insights into the past, present and latest trends in the automatic evaluation of summaries.

show abstract

Section: Task Based Evaluationmentioning

confidence: 99%

The challenging task of summary evaluation: an overview

Lloret

Plaza

Aker

2017

Lang Resources & Evaluation

View full text Add to dashboard Cite

show abstract

“…The systems that participate in Task B are given English questions written by biomedical experts that reflect real-life information needs. For each question, the systems are required to return relevant articles, snippets of the articles, concepts from designated ontologies, RDF triples from Linked Life Data, an 'exact' answer (e.g., a disease or symptom), and a paragraph-sized summary answer [1].…”

Section: Bioasqmentioning

confidence: 99%

Report on the Evaluation-as-a-Service (EaaS) Expert Workshop

Hopfgartner¹,

Hanbury²,

Müller³

et al. 2015

SIGIR Forum

Self Cite

View full text Add to dashboard Cite

In this report, we summarize the outcome of the "Evaluation-as-a-Service" workshop that was held on the 5th and 6th March 2015 in Sierre, Switzerland. The objective of the meeting was to bring together initiatives that use cloud infrastructures, virtual machines, APIs (Application Programming Interface) and related projects that provide evaluation of information retrieval or machine learning tools as a service.

show abstract

“…However, due to the need to assess technologies for automated identification and understanding of living organisms using data not only restricted to images, but also videos and sound, it was decided to be organised independently from ImageCLEF. Other CLEF labs linked to ImageCLEF, in particular the medical task, are: CLEFeHealth [10] that deals with processing methods and resources to enrich difficult-to-understand eHealth text and the BioASQ [3] tasks from the Question Answering lab that targets biomedical semantic indexing and question answering but is now not a lab anymore. Due to their medical topic, the organisation is coordinated in close collaboration with the medical tasks in ImageCLEF.…”

Section: Introductionmentioning

confidence: 99%

Overview of ImageCLEF 2017: Information Extraction from Images

Ionescu

Müller

Villegas³

et al. 2017

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Abstract. This paper presents an overview of the ImageCLEF 2017 evaluation campaign, an event that was organized as part of the CLEF (Conference and Labs of the Evaluation Forum) labs 2017. ImageCLEF is an ongoing initiative (started in 2003) that promotes the evaluation of technologies for annotation, indexing and retrieval for providing information access to collections of images in various usage scenarios and domains. In 2017, the 15th edition of ImageCLEF, three main tasks were proposed and one pilot task: 1) a LifeLog task about searching in LifeLog data, so videos, images and other sources; 2) a caption prediction task that aims at predicting the caption of a figure from the biomedical literature based on the figure alone; 3) a tuberculosis task that aims at detecting the tuberculosis type from CT (Computed Tomography) volumes of the lung and also the drug resistance of the tuberculosis; and 4) a remote sensing pilot task that aims at predicting population density based on satellite images. The strong participation of over 150 research groups registering for the four tasks and 27 groups submitting results shows the interest in this benchmarking campaign despite the fact that all four tasks were new and had to create their own community.

show abstract

BioASQ: A Challenge on Large-Scale Biomedical Semantic Indexing and Question Answering

Cited by 34 publications

References 23 publications

The challenging task of summary evaluation: an overview

The challenging task of summary evaluation: an overview

Report on the Evaluation-as-a-Service (EaaS) Expert Workshop

Overview of ImageCLEF 2017: Information Extraction from Images

Contact Info

Product

Resources

About