Patient representation learning and interpretable evaluation using clinical notes

Abstract

We have three contributions in this work: 1. We explore the utility of a stacked denoising autoencoder and a paragraph vector model to learn task-independent dense patient representations directly from clinical notes. To analyze if these representations are transferable across tasks, we evaluate them in multiple supervised setups to predict patient mortality, primary diagnostic and procedural category, and gender. We compare their performance with sparse representations obtained from a bag-of-words model. We observe that the learned generalized representations significantly outperform the sparse representations when we have few positive instances to learn from, and there is an absence of strong lexical features. 2. We compare the model performance of the feature set constructed from a bag of words to that obtained from medical concepts. In the latter case, concepts represent problems, treatments, and tests. We find that concept identification does not improve the classification performance. 3. We propose novel techniques to facilitate model interpretability. To understand and interpret the representations, we explore the best encoded features within the patient representations obtained from the autoencoder model. Further, we calculate feature sensitivity across two networks to identify the most significant input features for different classification tasks when we use these pretrained representations as the supervised input. We successfully extract the most influential features for the pipeline using this technique.

Combining structured and unstructured data for predictive models: a deep learning approach

“…Liu’s model [ 15 ] forecasts the onset of 3 kinds of diseases using medical notes. Sushil [ 16 ] utilizes a stacked denoised autoencoder and a paragraph vector model to learn generalized patient representation directly from clinical notes and the learned representation is used to predict mortality.…”

Representation learning for clinical time series prediction tasks in electronic health records

“…Miotto et al [25] adopted SDAs to generate patient representations. Furthermore, Sushil et al [26] derived task-independent patient representations directly from clinical notes by applying SDAs and a paragraph vector model. The above two methods only consider the frequency of medical events.…”

Toward a clinical text encoder: pretraining for clinical natural language processing with applications to substance misuse

“…The learned representations are used to predict ICD codes occurring in the next 30, 60, 90, and 180 days. In contrast to the previous works, Sushil et al 20 focuses exclusively on EHR text to learn patient representations using unsupervised methods, such as stacked denoising autoencoders and doc2vec. 7 They find that the learned representations outperform traditional bag-of-words representations when few training examples are available and that the target task does not rely on strong lexical features.…”

“…7 They find that the learned representations outperform traditional bag-of-words representations when few training examples are available and that the target task does not rely on strong lexical features. Like Sushil et al, 20 our work uses text variables only.…”