Critical assessment of transformer-based AI models for German clinical notes

Abstract

Objective Healthcare data such as clinical notes are primarily recorded in an unstructured manner. If adequately translated into structured data, they can be utilized for health economics and set the groundwork for better individualized patient care. To structure clinical notes, deep-learning methods, particularly transformer-based models like Bidirectional Encoder Representations from Transformers (BERT), have recently received much attention. Currently, biomedical applications are primarily focused on the English language. While general-purpose German-language models such as GermanBERT and GottBERT have been published, adaptations for biomedical data are unavailable. This study evaluated the suitability of existing and novel transformer-based models for the German biomedical and clinical domain. Materials and Methods We used 8 transformer-based models and pre-trained 3 new models on a newly generated biomedical corpus, and systematically compared them with each other. We annotated a new dataset of clinical notes and used it with 4 other corpora (BRONCO150, CLEF eHealth 2019 Task 1, GGPONC, and JSynCC) to perform named entity recognition (NER) and document classification tasks. Results General-purpose language models can be used effectively for biomedical and clinical natural language processing (NLP) tasks, still, our newly trained BioGottBERT model outperformed GottBERT on both clinical NER tasks. However, training new biomedical models from scratch proved ineffective. Discussion The domain-adaptation strategy’s potential is currently limited due to a lack of pre-training data. Since general-purpose language models are only marginally inferior to domain-specific models, both options are suitable for developing German-language biomedical applications. Conclusion General-purpose language models perform remarkably well on biomedical and clinical NLP tasks. If larger corpora become available in the future, domain-adapting these models may improve performances.

Year 2022 in Medical Natural Language Processing: Availability of Language Models as a Step in the Democratization of NLP in the Biomedical Area

“…Nevertheless, the trends observed within this set of keywords are also reflected in the analysis provided in the following sections. [23], construction of cohorts of similar patients [24], processing of electronic medical records [25], understanding of patient's answers in a French medical chatbot [26]; • German: evaluation of Transformers on clinical notes [27]; • Greek: improving the performance of localized healthcare virtual assistants [28]; • Hindi: classification of COVID-19 texts [29], chatbot for information sexual and reproductive health for young people [30]; • Italian: analysis of social media for quality of life in Parkinson's patients [31], sentiment analysis of opinion on COVID-19 vaccines [32,33], estimation of the incidence of infectious disease cases [34]; • Japanese: understanding psychiatric illness [35], detection of adverse events from narrative clinical documents [36]; • Korean: BERT model for processing med-ical documents [37], sentiment analysis of tweets about COVID-19 vaccines [38];…”

“…and institutions (like MIMIC-III), as well as data from social media, hospitals, bibliographical datasets, clinical trials, etc. The research in other languages is possible mainly thanks to the availability of data from social media [7,9,19,20,22,38,43,47] and documents from local hospitals [10,13,14,17,18,23,25,27,36,37,40,42]. Besides, this set of works in languages other than English relies on the dedicated language models, which cover a great variety of languages by now.…”

A distributable German clinical corpus containing cardiovascular clinical routine doctor’s letters

“…Therefore, shared corpora in the clinical domain are essential to support transparent and reproducible experiments and foster innovation in the field of clinical NLP 2 , 4 , 5 . In addition to their use for various medical information extraction tasks, e.g.…”

“…4 informatics for Life, Heidelberg, De, Germany. 5 Digital Health Center, Hasso Plattner Institute, University of Potsdam, Potsdam, DE, Germany. ✉ e-mail: phillip.richter-pechanski@med.uni-heidelberg.de Data DeSCRiPtoR oPeN 1. for medication information extraction and 2. for section classification.…”

Der Aufbruch generativer KI in der Medizin: Empathie durch Emulation

“…Dies ist insbesondere im Bereich der Medizin offensichtlich, wo diagnostische Algorithmen und Werkzeuge zur Bilderkennung ihr Potenzial gezeigt haben, die Erkennung und das Management von Krankheiten über menschliche Fähigkeiten hinaus voranzubringen [3] Die transformative Rolle von NLP ist besonders im Gesundheitswesen gefragt, wo eine Fülle von semi-und unstrukturierten Daten in Form von Anamnese, Arztberichten, medizinischen Unterla-gen und diagnostischen Berichten existiert. NLP kann diese, wenn auch unstrukturierten, Daten in strukturierte Datenpunkte umwandeln, die dann analysiert werden können, um handlungsrelevante Erkenntnisse zu gewinnen, die letztlich die Qualität und Effizienz der Patientenversorgung verbessern [10,11].…”