Supporting the Classification of Patients in Public Hospitals in Chile by Designing, Deploying and Validating a System Based on Natural Language Processing

Abstract

BackgroundIn Chile, a patient needing a specialty consultation or surgery has to first be referred by a general practitioner, then placed on a waiting list. The Explicit Health Guarantees (GES in Spanish) ensure, by law, the maximum time to solve an important set of health problems. Usually, a health professional manually verifies if each referral, written in natural language, corresponds or not to a GES-covered disease. An error in this classification is catastrophic for patients, as it puts them on a non-prioritized waiting list, characterized by prolonged waiting times. MethodsTo support the manual process, we developed and deployed a system that automatically classifies referrals as GES-covered or not using historical data. Our system is based on word embeddings specially trained for clinical text produced in Chile. We used a vector representation of the reason for referral and patient's age as features for training machine learning models using human-labeled historical data. We constructed a ground truth dataset combining classifications made by three healthcare experts, which was used to validate our results.ResultsThe best performing model over ground truth reached an AUC score of 0.94. During seven months of continuous and voluntary use, the system has amended 87 patient misclassifications.ConclusionThis system is a result of a collaboration between technical and clinical experts, and the design of the classifier was custom-tailored for a hospital's clinical workflow, which encouraged the voluntary use of the platform. Our solution can be easily expanded across other hospitals since the registry is uniform in Chile.

Construcción de recursos de texto para la identificación automática de información clínica en narrativas no estructuradas

“…En nuestro país transitamos una etapa incipiente del PLN, y una pieza fundamental en la colaboración entre equipos clínicos e informáticos es la de producir insumos claves para el entrenamiento de algoritmos en los que las máquinas puedan apoyar la decisión clínica. La colaboración interdisciplinaria ofrece una sinergia cuyo impacto positivo en la optimización de procesos clínicos y administrativos en salud pública, lo que se traduce en una mayor calidad y justicia en el sistema de salud 29 .…”

Clinical Flair: A Pre-Trained Language Model for Spanish Clinical Natural Language Processing

“…The Chilean Waiting List corpus consists of 5, 157, 902 free-text diagnostic suspicions comprising 14, 057, 401 sentences and 68, 541, 727 tokens. Although the general purpose of this dataset was to be a new resource for named entity recognition, it has also been used to obtain static word embeddings from the clinical domain (Villena et al, 2021b). These representations have boosted the model's performance in several clinical NLP tasks such as tumor encoding (Villena et al, 2021a) and named entity recognition (Báez et al, 2022).…”

Proceedings of the 4th Clinical Natural Language Processing Workshop

“…The Chilean Waiting List corpus consists of 5, 157, 902 free-text diagnostic suspicions comprising 14, 057, 401 sentences and 68, 541, 727 tokens. Although the general purpose of this dataset was to be a new resource for named entity recognition, it has also been used to obtain static word embeddings from the clinical domain (Villena et al, 2021b). These representations have boosted the model's performance in several clinical NLP tasks such as tumor encoding (Villena et al, 2021a) and named entity recognition (Báez et al, 2022).…”