Terminologies augmented recurrent neural network model for clinical named entity recognition

Abstract

ObjectiveWe aimed to enhance the performance of a supervised model for clinical named-entity recognition (NER) using medical terminologies. In order to evaluate our system in French, we built a corpus for 5 types of clinical entities. MethodsWe used a terminology-based system as baseline, built upon UMLS and SNOMED. Then, we evaluated a biGRU-CRF, and an hybrid system using the prediction of the terminology-based system as feature for the biGRU-CRF. In English, we evaluated the NER systems on the i2b2-2009 Medication Challenge for Drug name recognition, which contained 8,573 entities for 268 documents . In French, we built APcNER, a corpus of 147 documents annotated for 5 entities (drug name, sign or symptom, disease or disorder, diagnostic procedure or lab test and therapeutic procedure). We evaluated each NER systems using exact and partial match definition of F-measure for NER. ResultsThe APcNER contains 4,837 entities which took 28 hours to annotate, the inter-annotator agreement was acceptable for Drug name in exact match (85%) and acceptable for other entity types in non-exact match (>70%). For drug name recognition on both i2b2-2009 and APcNER, the biGRU-CRF performed better that the terminology-based system, with an exact-match F-measure of 91.1% versus 73% and 81.9% versus 75% respectively. Moreover, the hybrid system outperformed the biGRU-CRF, with an exact-match F-measure of 92.2% versus 91.1% (i2b2-2009) and 88.4% versus 81.9% (APcNER). On APcNER corpus, the micro-average F-measure of the hybrid system on the 5 entities was 69.5% in exact match, and 84.1% in non-exact match. ConclusionAPcNER is a French corpus for clinical-NER of five type of entities which covers a large variety of document types. Extending supervised model with terminology allowed for an easy performance gain, especially in low regimes of entities, and established near state of the art results on the i2b2-2009 corpus.

Hybrid Deep Learning for Medication-Related Information Extraction From Clinical Texts in French: MedExt Algorithm Development Study

“…Moreover, we took the condition of the intake, and not the reason for the intake, into consideration (which is more specific), and we added a tag regarding the class name; therefore, overall F-measures cannot be compared. Compared with results from a study [33] using a different French-language corpus that obtained a token-level F-measure of 90.4, our system's raw results were higher. Comparisons should be made with caution because the corpus used in [33], though in the same language, was from a different source and contained only 147 documents.…”

“…Compared with results from a study [33] using a different French-language corpus that obtained a token-level F-measure of 90.4, our system's raw results were higher. Comparisons should be made with caution because the corpus used in [33], though in the same language, was from a different source and contained only 147 documents.…”

“…These approaches were designed for text written in English. To the best of our knowledge, there are only a few studies [32,33] on French corpora: Deleger et al [32] used a rule-based system, and Lerner et al [33] developed a hybrid system that associated expert rules using terminology and bidirectional gated recurrent units with a conditional random field.…”

A Year of Papers Using Biomedical Texts:

“…French was the third language (seven mentions) as in the work by Lerner et al 6 , followed by three other European languages with less than five mentions: German, Italian 7 , and Spanish 8 ;…”

“…Yet, we can consider that the papers that did not explicitly indicate the language should also be dedicated to the processing of data in English ; • Chinese became the second language processed in medical NLP papers with 17 mentions. Among the papers published in 2019, we can mention Guan et al [3] working on the generation of synthetic medical record texts, Chen et al [4] aiming at identifying named entities, and Zheng et al [5] interested by the detection of medical text similarity ; • French was the third language (seven mentions) as in the work by Lerner et al [6], followed by three other European languages with less than five mentions: German, Italian [7], and Spanish [8] ; • Other languages identified in the abstracts accounted for one or two papers and included both languages spoken by millions of people (Arabic, Portuguese, Russian) and languages spoken by small communities (Basque, Danish, Japanese, Korean, Lithuanian, Persian, Romanian, Turkish, and Urdu).…”