Identifying Alcohol-Related Information From Unstructured Bilingual Clinical Notes With Multilingual Transformers

Abstract

As a key modifiable risk factor, alcohol consumption is clinically crucial information that allows medical professionals to further understand their patients' medical conditions and suggest appropriate lifestyle modifying interventions. However, identifying alcohol-related information from unstructured freetext clinical notes is often challenging. Not only are the formats of the notes inconsistent, but they also include a massive amount of non-alcohol-related information. Furthermore, for medical institutions outside of English-speaking countries, these clinical notes contain both a mixture of English and local languages, inducing additional difficulty in the extraction. Thanks to the increasing availability of electronic medical record (EMR), several previous works explored the idea of using natural language processing (NLP) to train machine learning models that automatically identify alcohol-related information from unstructured clinical notes. However, all these previous works are limited to English clinical notes, thereby able to leverage various large-scale external ontologies during the text preprocessing. Furthermore, they rely on simple NLP techniques such as the bag-of-words models that suffer from high dimensionality and out-ofvocabulary issues. Addressing these issues, we adopt fine-tuning multilingual transformers. By leveraging their linguistically rich contextual information learned during their pre-training, we are able to extract alcohol-related information from unstructured clinical notes without preprocessing the clinical notes on any external ontologies. Furthermore, our work is the first to explore the use of transformers in bilingual clinical notes to extract alcohol-related information. Even with minimal text preprocessing, we achieve extraction accuracy of 84.70% in terms of macro F-1 score. INDEX TERMS clinical informatics, alcohol information extraction, natural language processing, information extraction from clinical notes, multilingual transformers I. INTRODUCTIONAs medical institutions worldwide are widely adopting electronic medical record (EMR), vast amounts of healthcare data are produced and stored electronically [1-5]. As a significant component of EMR, clinical notes, which record patients' conditions in free text, provide essential information such as patients' medical history, social history, or lifestyle patterns. Despite being a vital data source, its practical use in medical decision support systems is hampered by challenges in extracting key information from its unstructured text format [6][7][8][9]. For medical institutions in non-English speaking countries, these challenges are compounded by the use of both English and local languages in their clinical notes. Besides standardizing the notes to resolve any inconsistent formats or structures, they need to handle the multilingual aspect of their notes simultaneously. Due to this additional