The Scholarly Impact of CLEF (2000–2009)

Tsikrika, Theodora; Larsen, Birger; Müller, Henning; Endrullis, Stefan; Rahm, Erhard

doi:10.1007/978-3-642-40802-1_1

Cited by 28 publications

(21 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The scholarly influence of the CLEF eHealth installations was measured by conducting a bibliometric study -an established method to provide a quantitative and qualitative indication of scientific activities whose use is also emerging in the context of evaluation initiatives [9][10][11] -of the publications generated as a result of these installations in 2012-2017 and their citations received by 31 October 2017.…”

Section: Methodsmentioning

confidence: 99%

“…Citation data for the resulting publication data was collected from Google Scholarone of the most comprehensive citation data sources in general and in particular for computer science, which is the main field of many CLEF eHealth scientists [9][10][11].…”

Section: Methodsmentioning

confidence: 99%

“…Its results were supplemented by the known journal extensions of the CLEF eHealth overviews and previously enlisted papers that use the CLEF eHealth datasets [8]. WNs were technical reports written by participants describing their participation in the lab.Citation data for the resulting publication data was collected from Google Scholarone of the most comprehensive citation data sources in general and in particular for computer science, which is the main field of many CLEF eHealth scientists [9][10][11].The method used for the data analysis of the third step was the citation content analysis [24], founded on the content analysis [25] and grounded theory [26]. This allowed a systematic, replicable compression of materials from the first and second step as codes and testing of hypotheses about both the quantity and quality of the scholarly influence of CLEF eHealth in 2012-2017.…”

mentioning

confidence: 99%

“…both expression of interest (EOI and submission), author, affiliation, problem specification, evaluation method, benchmark result, data release, software launch, demonstration system, and citation. Similarly to [10], attention was paid to not only the number of citations, but also the number of authors, their affiliations, and affiliation countries. In order to illustrate the influence not only to the scholarly community but also to the individual scholars (because most participating teams included graduate students and/or early career academics), the scholarly influence (i.e., 962,559 = 1,299 × 741) was computed by multiplying the number of citations (i.e., 1,299) for the included papers (i.e., 184) by the number of their co-authors (i.e., 741).…”

mentioning

confidence: 99%

See 3 more Smart Citations

Scholarly Influence of the Conference and Labs of the Evaluation Forum eHealth Initiative: Review and Bibliometric Study of the 2012 to 2017 Outcomes (Preprint)

Suominen¹,

Kelly²,

Goeuriot³

2018

Preprint

View full text Add to dashboard Cite

Background: The eHealth initiative of the Conference and Labs of the Evaluation Forum (CLEF) has aimed since 2012 to gather researchers working on health text analytics and to provide them with annual workshop, shared development challenges/tasks, benchmark datasets, and software for processing and evaluation. The overall purpose of this initiative is to ease and support patients, their next-ofkin, clinical staff, health scientists, and healthcare policy makers in accessing, understanding, using, and authoring health information in a multilingual setting. Objective: This original research paper reports on the outcomes of the first six installations of CLEF eHealth from 2012 to 2017. The focus is on measuring and analysing the scholarly influence by reviewing CLEF eHealth papers, together with relevant citation metrics. Methods: A review and bibliometric study of the CLEF eHealth proceedings, working notes, and author-declared paper extensions was conducted. Citation data for these publications were collected from Google Scholar. Citation content analysis was used for the publications and their citations. Results: The large number of registrations, submissions, and citations demonstrate the substantial community interest in the tasks and their resources. In total, 718 teams have registered their interest in the tasks, leading to 130 teams submitting to the 15 tasks. 184 papers using CLEF eHealth data generated 1,299 citations, yielding a total scholarly citation influence of almost 963,000 citations for the 741 co-authors and included authors from 33 countries across the world. The tasks' evaluation outcomes contribute to the knowledge of the difficulty of the research challenges the tasks address and the applicability of particular methods in solving these challenges, with typically statistically significant improvements in processing quality. Conclusions: These outcomes encourage continuing to develop these technologies to address patient needs. Consequently, data and tools have been opened for future research and development and the CLEF eHealth initiative continues to run new challenges. Keywords: Evaluation Studies as Topic; Health Records; Information Extraction; Information Storage and Retrieval; Information Visualization; Patient Education as Topic; Speech Recognition; Systematic Reviews; Test-set Generation; Text Classification IntroductionHealth information refers to all health-related content in all data formats, document types, information systems, publication media, and languages from all organisations, states, and countries. The privacy-sensitive, official part of health information consists of data recorded in healthcare services when describing a given patient's health or healthcare (Figure 1). The accessibility of this data is defined as limited (i.e., private or confidential information) and it is recorded either on paper in health records or electronically in Electronic Health (eHealth) records. Some common synonyms or related terms include eHealth charts, data, documents, information, ...

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Methodsmentioning

confidence: 99%

mentioning

confidence: 99%

mentioning

confidence: 99%

See 2 more Smart Citations

Scholarly Influence of the Conference and Labs of the Evaluation Forum eHealth Initiative: Review and Bibliometric Study of the 2012 to 2017 Outcomes (Preprint)

Suominen¹,

Kelly²,

Goeuriot³

2018

Preprint

View full text Add to dashboard Cite

show abstract

“…Unfortunately, the percentage of groups that submit results is often relatively small. Nevertheless, as observed in studies of scholarly impact [22,23], in subsequent years the datasets and challenges provided by ImageCLEF do get used quite often, which in part is due to the researchers that for some reason were unable to participate in the original event.…”

Section: Overview Of Tasks and Participationmentioning

confidence: 99%

Overview of ImageCLEF 2017: Information Extraction from Images

Ionescu

Müller

Villegas³

et al. 2017

Lecture Notes in Computer Science

Self Cite

View full text Add to dashboard Cite

Abstract. This paper presents an overview of the ImageCLEF 2017 evaluation campaign, an event that was organized as part of the CLEF (Conference and Labs of the Evaluation Forum) labs 2017. ImageCLEF is an ongoing initiative (started in 2003) that promotes the evaluation of technologies for annotation, indexing and retrieval for providing information access to collections of images in various usage scenarios and domains. In 2017, the 15th edition of ImageCLEF, three main tasks were proposed and one pilot task: 1) a LifeLog task about searching in LifeLog data, so videos, images and other sources; 2) a caption prediction task that aims at predicting the caption of a figure from the biomedical literature based on the figure alone; 3) a tuberculosis task that aims at detecting the tuberculosis type from CT (Computed Tomography) volumes of the lung and also the drug resistance of the tuberculosis; and 4) a remote sensing pilot task that aims at predicting population density based on satellite images. The strong participation of over 150 research groups registering for the four tasks and 27 groups submitting results shows the interest in this benchmarking campaign despite the fact that all four tasks were new and had to create their own community.

show abstract

ImageCLEF 2019: Multimedia Retrieval in Lifelogging, Medical, Nature, and Security Applications

Ionescu

Müller

Péteri

et al. 2019

Lecture Notes in Computer Science

Self Cite

View full text Add to dashboard Cite

This paper presents an overview of the foreseen ImageCLEF 2019 lab that will be organized as part of the Conference and Labs of the Evaluation Forum-CLEF Labs 2019. ImageCLEF is an ongoing evaluation initiative (started in 2003) that promotes the evaluation of technologies for annotation, indexing and retrieval of visual data with the aim of providing information access to large collections of images in various usage scenarios and domains. In 2019, the 17th edition of Image-CLEF will run four main tasks: (i) a Lifelog task (videos, images and other sources) about daily activities understanding, retrieval and summarization, (ii) a Medical task that groups three previous tasks (caption analysis, tuberculosis prediction, and medical visual question answering) with newer data, (iii) a new Coral task about segmenting and labeling collections of coral images for 3D modeling, and (iv) a new Security task addressing the problems of automatically identifying forged content

show abstract

The Scholarly Impact of CLEF (2000–2009)

Cited by 28 publications

References 8 publications

Scholarly Influence of the Conference and Labs of the Evaluation Forum eHealth Initiative: Review and Bibliometric Study of the 2012 to 2017 Outcomes (Preprint)

Scholarly Influence of the Conference and Labs of the Evaluation Forum eHealth Initiative: Review and Bibliometric Study of the 2012 to 2017 Outcomes (Preprint)

Overview of ImageCLEF 2017: Information Extraction from Images

ImageCLEF 2019: Multimedia Retrieval in Lifelogging, Medical, Nature, and Security Applications

Contact Info

Product

Resources

About