Overview of the CLEF eHealth Evaluation Lab 2018

Abstract

In this paper, we provide an overview of the sixth annual edition of the CLEF eHealth evaluation lab. CLEF eHealth 2018 continues our evaluation resource building efforts around the easing and support of patients, their next-of-kins, clinical staff, and health scientists in understanding, accessing, and authoring eHealth information in a multilingual setting. This year's lab offered three tasks: Task 1 on multilingual information extraction to extend from last year's task on French and English corpora to French, Hungarian, and Italian; Task 2 on technologically assisted reviews in empirical medicine building on last year's pilot task G. Zuccon-In alphabetical order by forename, HS, LK & LG co chaired the lab.

CLEF eHealth Evaluation Lab 2020

“…CLEF eHealth tasks offered yearly from 2013 have brought together researchers working on related information access topics, provided them with resources to work with and validate their outcomes, and accelerated pathways from scientific ideas to societal impact. In 2013, 2014, 2015, 2016, 2017, 2018, and 2019 as many as 170, 220, 100, 116, 67, 70, and 67 teams have registered their expression of interest in the CLEF eHealth tasks, respectively, and the number of teams proceeding to the task submission stage has been 53, 24, 20, 20, 32, 28, and 9, respectively [4,5,[8][9][10]16,17]. 2 According to our analysis of the impact of CLEF eHealth labs up to 2017 [15], the submitting teams have achieved statistically significant improvements in the processing quality in at least 1 out of the top-3 methods submitted to the following eight tasks: 3 The 2012-2017 contributions have been reported by October 2018 in 184 papers for the 741 included authors from 33 countries across the world, and the papers have attracted nearly 1, 300 citations, creating h-index and i10-index of 18 and 35, respectively, on Google Scholar [14].…”

“…The 2020 CLEF eHealth Task 1 on IE, called CodiEsp supported by the Spanish National Plan for the Advancement of Language Technology (Plan TL), builds upon the five previous editions of the task in 2015-2019 [4,5,8,10,16] that have already addressed the analysis of biomedical text in English, French, Hungarian, Italian, and German. This year, the CodiEsp task, will focus on the International Classification of Diseases, the 10th Revision (ICD10) coding for clinical case data in Spanish using the Spanish version of ICD10 (CIE10).…”

“…Yet, with rare exception, they have targeted the health care experts' information needs only [1,2,6]. Such exception, the CLEF eHealth Evaluation-lab and Lab-workshop Series 1 has been organized every year since 2012 as part of the Conference and Labs of the Evaluation Forum (CLEF) [4,5,[8][9][10]13,16,17]. In 2012, the inaugural scientific CLEF workshop took place, and from 2013-2019 this annual workshop has been supplemented with a lead-up evaluation lab, consisting of, on average, three shared tasks each year ( Fig.…”

CLEF eHealth 2019 Evaluation Lab

“…In total, 70 unique teams registered their interest and 28 teams took part in the 2018 tasks (14 in Task 1, 7 in Task 2 and 7 in Task 3). In comparison, in 2017, 2016, 2015, 2014, and 2013, the number of team registrations was 67, 116, 100, 220, and 175, respectively and the number of participating teams was 32, 20, 20, 24, and 53 [20,11,5,10,6,19].…”

“…In this paper we overview the CLEF eHealth evaluation lab series to-date [20,11,5,10,6,19] and present this year's evaluation lab challenges.…”