Z-LaVI: Zero-Shot Language Solver Fueled by Visual Imagination

A two-stream deep model for automated ICD-9 code prediction in an intensive care unit

“…In addition, although the maximum number of tokens was increased to 1024 with two Clinical BERT models for vectorization of discharge summaries, the results were insufficient. This inference is also similar to the conclusion in Yang et al's [7] study. Moreover, Biswas et al [28] , Zhang et al [29] , Pascual et al [30] also showed the necessity of using higher dimensional token vectors.…”

“…For text-based classification, state-of-the-art transformer-based NLP methods were utilized and evaluated. These methods include Clinical BERT [14] , Clinical Longformer [24] and KEPT [7] based methods, respectively.…”

“…To generalize the results obtained in this study, experiments were also carried out using the KEPT method, which has the most successful result in the literature for the top 50 ICD-9 prediction tasks. The KEPT method is another Longformer-based language model proposed by Yang et al [7] for ICD encoding purposes. In this method, an alternative approach is developed for multi-class tasks.…”

“…Dataset B: Xie et al [3] arranged the discharge reports into training, validation, and test datasets based on 8066, 1728, and 1729 patient referrals from the MIMIC-III dataset, respectively. Afterward, this dataset was used in various state-of-the-art studies including [4] , [5] , [6] , [7] . In this study, this dataset is referred to as Dataset B.…”

“…Yang et al proposed a deep learning method based on the Longformer method, powered by the Knowledge Enhanced PrompT (KEPT) method [7] . They explained that they chose a Longformer-based method as the natural language processing (NLP) method because the texts in the MIMIC-III dataset were too large to be fully represented by other methods.…”