A Predictive Model for Medical Events Based on Contextual Embedding of Temporal Sequences

Abstract

BackgroundMedical concepts are inherently ambiguous and error-prone due to human fallibility, which makes it hard for them to be fully used by classical machine learning methods (eg, for tasks like early stage disease prediction).ObjectiveOur work was to create a new machine-friendly representation that resembles the semantics of medical concepts. We then developed a sequential predictive model for medical events based on this new representation.MethodsWe developed novel contextual embedding techniques to combine different medical events (eg, diagnoses, prescriptions, and labs tests). Each medical event is converted into a numerical vector that resembles its “semantics,” via which the similarity between medical events can be easily measured. We developed simple and effective predictive models based on these vectors to predict novel diagnoses.ResultsWe evaluated our sequential prediction model (and standard learning methods) in estimating the risk of potential diseases based on our contextual embedding representation. Our model achieved an area under the receiver operating characteristic (ROC) curve (AUC) of 0.79 on chronic systolic heart failure and an average AUC of 0.67 (over the 80 most common diagnoses) using the Medical Information Mart for Intensive Care III (MIMIC-III) dataset.ConclusionsWe propose a general early prognosis predictor for 80 different diagnoses. Our method computes numeric representation for each medical event to uncover the potential meaning of those events. Our results demonstrate the efficiency of the proposed method, which will benefit patients and physicians by offering more accurate diagnosis.

Privacy-Preserving Predictive Modeling: Harmonization of Contextual Embeddings From Different Sources (Preprint)

“…For this paper, we choose the skipgram model for its simplicity and efficiency. The model requires two parameters, size and window, defining the dimensionality of the final vector representation and maximum distance for contextual consideration, respectively [20,23]. There is one limitation to contextual embedding techniques such as Word2Vec and GloVe [24].…”

“…To predict the next likely diagnosis of a new patient for structured data experiments, we use patient-diagnosis projection similarity (PDPS) method [20]. To calculate PDPS, we first create a patient vector.…”

“…kidney failure). Models that utilize such representation have shown higher prediction performance than previous models that do not [20,21].…”

“…Recently, there has been considerable attention in the application of neural networks to represent medical concepts as multi-dimensional and continuous vectors [20,21]. A process called contextual embedding, commonly used in natural language processing, maps each word from a corpus of text to a hyper-dimensional space where similar words in terms of meaning and/or distributed usage would be located nearby (e.g., short cosine distance).…”

Health-ATM: A Deep Architecture for Multifaceted Patient Health Record Representation and Risk Prediction

“…• Time-aware convolution: Time stamps are already demonstrated useful in predicting future events [14]. For example, Choi et al [8] integrated time information into their RNN based models to predict the occurrence and timing of near-term subsequent events.…”