Medical Coding with Biomedical Transformer Ensembles and Zero/Few-shot Learning

Ziletti, Angelo; Akbik, Alan; Berns, Christoph; Herold, Thomas; Marion, Legler,; Martina, Viell,

doi:10.18653/v1/2022.naacl-industry.21

Cited by 2 publications

(2 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The number of code representations stored in memory increases linearly as the number of candidate codes to assign increases. The significant increase in memory makes it hard to deploy in the real-world auto ICD classification setting (Ziletti et al 2022;Yan et al 2022).…”

Section: Code Frequencymentioning

confidence: 99%

Multi-Label Few-Shot ICD Coding as Autoregressive Generation with Prompt

Yang

Kwon

Yao

et al. 2023

AAAI

View full text Add to dashboard Cite

Automatic International Classification of Diseases (ICD) coding aims to assign multiple ICD codes to a medical note with an average of 3,000+ tokens. This task is challenging due to the high-dimensional space of multi-label assignment (155,000+ ICD code candidates) and the long-tail challenge - Many ICD codes are infrequently assigned yet infrequent ICD codes are important clinically. This study addresses the long-tail challenge by transforming this multi-label classification task into an autoregressive generation task. Specifically, we first introduce a novel pretraining objective to generate free text diagnosis and procedure descriptions using the SOAP structure, the medical logic physicians use for note documentation. Second, instead of directly predicting the high dimensional space of ICD codes, our model generates the lower dimension of text descriptions, which then infer ICD codes. Third, we designed a novel prompt template for multi-label classification. We evaluate our Generation with Prompt (GP) model with the benchmark of all code assignment (MIMIC-III-full) and few shot ICD code assignment evaluation benchmark (MIMIC-III-few). Experiments on MIMIC-III-few show that our model performs with a marco F1 30.2, which substantially outperforms the previous MIMIC-III-full SOTA model (marco F1 4.3) and the model specifically designed for few/zero shot setting (marco F1 18.7). Finally, we design a novel ensemble learner, a cross attention reranker with prompts, to integrate previous SOTA and our best few-shot coding predictions. Experiments on MIMIC-III-full show that our ensemble learner substantially improves both macro and micro F1, from 10.4 to 14.6 and from 58.2 to 59.1, respectively.

show abstract

Section: Code Frequencymentioning

confidence: 99%

Multi-Label Few-Shot ICD Coding as Autoregressive Generation with Prompt

Yang

Kwon

Yao

et al. 2023

AAAI

View full text Add to dashboard Cite

show abstract

“…Medical text classification is widely recognized as an urgent yet challenging problem due to its extremely imbalanced data distribution, large variety of rare labels (Johnson et al, 2016;Ziletti et al, 2022), and complicated label relationship (Tsai et al, 2021;Vu et al, 2021). Various downstream clinical tasks have been derived from this problem, including ICD coding (Mullenbach et al, 2018;Yuan et al, 2022;Yang et al, 2022) and automated diagnosis (Chen et al, 2020b), showcasing its potential values in modern clinical practice with machine learning approaches (Berner, 2007).…”

Section: Introductionmentioning

confidence: 99%

Text2Tree: Aligning Text Representation to the Label Tree Hierarchy for Imbalanced Medical Classification

Yan,

Gao,

Kai

et al. 2023

Findings of the Association for Computational Linguistics: EMNLP 2023

View full text Add to dashboard Cite

Deep learning approaches exhibit promising performances on various text tasks. However, they are still struggling on medical text classification since samples are often extremely imbalanced and scarce. Different from existing mainstream approaches that focus on supplementary semantics with external medical information, this paper aims to rethink the data challenges in medical texts and present a novel framework-agnostic algorithm called Text2Tree that only utilizes internal label hierarchy in training deep learning models. We embed the ICD code tree structure of labels into cascade attention modules for learning hierarchy-aware label representations. Two new learning schemes, Similarity Surrogate Learning (SSL) and Dissimilarity Mixup Learning (DML), are devised to boost text classification by reusing and distinguishing samples of other labels following the label representation hierarchy, respectively. Experiments on authoritative public datasets and real-world medical records show that our approach stably achieves superior performances over classical and advanced imbalanced classification methods. Our code is available at https://github.com/jyansir/Text2Tree.

show abstract

Medical Coding with Biomedical Transformer Ensembles and Zero/Few-shot Learning

Cited by 2 publications

References 14 publications

Multi-Label Few-Shot ICD Coding as Autoregressive Generation with Prompt

Multi-Label Few-Shot ICD Coding as Autoregressive Generation with Prompt

Text2Tree: Aligning Text Representation to the Label Tree Hierarchy for Imbalanced Medical Classification

Contact Info

Product

Resources

About