Entity-Centric Information Access with the Human-in-the-Loop for the Biomedical Domains

Yimam, Seid Muhie; Remus, Steffen; Panchenko, Alexander; Holzinger, Andreas; Biemann, Chris

doi:10.26615/978-954-452-044-1_006

Cited by 6 publications

(3 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Criterion P1 is also important for tracking future citations to the tool, and thus for enforcing its credibility. The tools that obtained the highest evaluation for all three publication criteria were brat and WebAnno, which are very popular and has already been used for a variety of corpus constructions, also in the biomedical domain [ 91 ].…”

Section: Discussionmentioning

confidence: 99%

An extensive review of tools for manual annotation of documents

Neves

Ševa

2019

Briefings in Bioinformatics

View full text Add to dashboard Cite

Motivation Annotation tools are applied to build training and test corpora, which are essential for the development and evaluation of new natural language processing algorithms. Further, annotation tools are also used to extract new information for a particular use case. However, owing to the high number of existing annotation tools, finding the one that best fits particular needs is a demanding task that requires searching the scientific literature followed by installing and trying various tools. Methods We searched for annotation tools and selected a subset of them according to five requirements with which they should comply, such as being Web-based or supporting the definition of a schema. We installed the selected tools (when necessary), carried out hands-on experiments and evaluated them using 26 criteria that covered functional and technical aspects. We defined each criterion on three levels of matches and a score for the final evaluation of the tools. Results We evaluated 78 tools and selected the following 15 for a detailed evaluation: BioQRator, brat, Catma, Djangology, ezTag, FLAT, LightTag, MAT, MyMiner, PDFAnno, prodigy, tagtog, TextAE, WAT-SL and WebAnno. Full compliance with our 26 criteria ranged from only 9 up to 20 criteria, which demonstrated that some tools are comprehensive and mature enough to be used on most annotation projects. The highest score of 0.81 was obtained by WebAnno (of a maximum value of 1.0).

show abstract

Section: Discussionmentioning

confidence: 99%

An extensive review of tools for manual annotation of documents

Neves

Ševa

2019

Briefings in Bioinformatics

View full text Add to dashboard Cite

show abstract

“…A conflict between transparency and performance of an algorithm becomes apparent [ 60 ]. One way to use more powerful models and ensure explainability is a human-in-the-loop approach [ 61 ]. In this approach, the results are presented to and monitored by an expert.…”

Section: Use Case B: Inpatient Information Management With Automated ...mentioning

confidence: 99%

Accelerating the Front End of Medicine: Three Digital Use Cases and HCI Implications

et al. 2022

View full text Add to dashboard Cite

Digital applications in health care are a concurrent research and management question, where implementation experiences are a core field of information systems research. It also contributes to fighting pandemic crises like COVID-19 because contactless information flow and speed of diagnostics are improved. This paper presents three digital application case studies from emergency medicine, administration management, and cancer diagnosis with AI support from the University Medical Centers of Münster and Göttingen in Germany. All cases highlight the potential of digitalization to increase speed and efficiency within the front end of medicine as the crucial phase before patient treatment starts. General challenges for health care project implementations and human-computer interaction (HCI) concepts in health care are derived and discussed, including the importance of specific processes together with user analysis and adaption. A derived concept for HCI includes the criteria speed, accuracy, modularity, and individuality to achieve sustainable improvements within the front end of medicine.

show abstract

“…Professionals explore pre-annotated data using a faceted search interface and can add annotations. E.g., HiL is being employed in the biomedical domain [21], which needs an entity-centric access (bottom-up). In our use-case, we are concentrating on revision outcomes (top-down classification), that can be explored by different meta information.…”

Section: Related Workmentioning

confidence: 99%

LawStats – Large-Scale German Court Decision Evaluation Using Web Service Classifiers

Ruppert

Hartung

Sittig

et al. 2018

Lecture Notes in Computer Science

Self Cite

View full text Add to dashboard Cite

LawStats provides quantitative insights into court decisions from the Bundesgerichtshof -Federal Court of Justice (BGH), the Federal Court of Justice in Germany. Using Watson Web Services and approaches from Sentiment Analysis (SA), we can automatically classify the revision outcome and offer statistics on judges, senates, previous instances etc. via faceted search. These statistics are accessible through a open web interface to aid law professionals. With a clear focus on interpretability, users can not only explore statistics, but can also understand, which sentences in the decision are responsible for the machine's decision; links to the original texts provide more context. This is the first largescale application of Machine Learning (ML) based Natural Language Processing (NLP) for German in the analysis of ordinary court decisions in Germany that we are aware of. We have analyzed over 50,000 court decisions and extracted the outcomes and relevant entities. The modular architecture of the application allows continuous improvements of the ML model as more annotations become available over time. The tool can provide a critical foundation for further quantitative research in the legal domain and can be used as a proof-of-concept for similar efforts.

show abstract

Entity-Centric Information Access with the Human-in-the-Loop for the Biomedical Domains

Cited by 6 publications

References 18 publications

An extensive review of tools for manual annotation of documents

An extensive review of tools for manual annotation of documents

Accelerating the Front End of Medicine: Three Digital Use Cases and HCI Implications

LawStats – Large-Scale German Court Decision Evaluation Using Web Service Classifiers

Contact Info

Product

Resources

About