On the Properties of Neural Machine Translation: Encoder-Decoder Approaches

Cho, Kyunghyun; Merriënboer, Bart van; Bahdanau, Dzmitry; Bengio, Yoshua

doi:10.48550/arxiv.1409.1259

Cited by 1,072 publications

(1,179 citation statements)

References 1 publication

Supporting

Mentioning

1,014

Contrasting

Order By: Relevance

“…We incentivised our model to recognise that a sepsis will start within the next 6 h. To improve upon clinical baselines, we investigated two families of classifiers: deep learning approaches and non-deep ML approaches. As for deep models, we considered a self-attention model (attn) [41] as well as a recurrent neural network employing Gated Recurrent Units (gru) [5], both of which are intrinsically capable of leveraging sequential data. Next, we included LightGBM (lgbm) [17] and a LASSO-regularised Logistic regression (lr) [39], which were given access to a total of 1,269 features that were extracted in order to make temporal dynamics governing the data accessible to these methods.…”

Section: Resultsmentioning

confidence: 99%

See 1 more Smart Citation

Predicting sepsis in multi-site, multi-national intensive care cohorts using deep learning

Moor,

Bennet,

Plecko

et al. 2021

Preprint

View full text Add to dashboard Cite

Despite decades of clinical research, sepsis remains a global public health crisis with high mortality, and morbidity. Currently, when sepsis is detected and the underlying pathogen is identified, organ damage may have already progressed to irreversible stages. Effective sepsis management is therefore highly time-sensitive. By systematically analysing trends in the plethora of clinical data available in the intensive care unit (ICU), an early prediction of sepsis could lead to earlier pathogen identification, resistance testing, and effective antibiotic and supportive treatment, and thereby become a life-saving measure. Here, we developed and validated a machine learning (ML) system for the prediction of sepsis in the ICU. Our analysis represents the largest multi-national, multi-centre in-ICU study for sepsis prediction using ML to date. Our dataset contains 156, 309 unique ICU admissions, which represent a refined and harmonised subset of five large ICU databases originating from three countries. Using the international consensus definition Sepsis-3, we derived hourly-resolved sepsis label annotations, amounting to 26, 734 (17.1%) septic stays. We compared our approach, a deep self-attention model, to several clinical baselines as well as ML baselines and performed an extensive internal and external validation within and across databases. On average, our model was able to predict sepsis with an AUROC of 0.847 ± 0.050 (internal out-of sample validation) and 0.761 ± 0.052 (external validation). For a harmonised prevalence of 17%, at 80% recall our model detects septic patients with 39% precision 3.7 h in advance.

show abstract

Section: Resultsmentioning

confidence: 99%

“…In this study, we investigated a comprehensive selection of supervised ML approaches. This includes i) deep self-attention models (attn) [41] ii) recurrent neural networks employing gated recurrent units (gru) [5] iii) LightGBM gradient boosting trees (lgbm) [17], and iv) LASSO-regularised [39] logistic regression (lr).…”

Section: Methodsmentioning

confidence: 99%

Predicting sepsis in multi-site, multi-national intensive care cohorts using deep learning

Moor,

Bennet,

Plecko

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…We then performed 7 × 7 max pooling with a stride of 5 × 5. The output of the CNN was reshaped and provided as input to an RNN with a gated recurrent unit Cho et al [ 51 ] model of size 128, followed by a fully connected layer. We used the partial fine-tuning approach [ 52 ] for the tuning the CNN component, where only the affine weights of the batch normalisation layers are updated while the rest of the weights in the CNN remain frozen.…”

Section: Methodsmentioning

confidence: 99%

A Deep Learning Model for Cervical Cancer Screening on Liquid-Based Cytology Specimens in Whole Slide Images

Kanavati

Hirose

Ishii

et al. 2022

Cancers

View full text Add to dashboard Cite

Liquid-based cytology (LBC) for cervical cancer screening is now more common than the conventional smears, which when digitised from glass slides into whole-slide images (WSIs), opens up the possibility of artificial intelligence (AI)-based automated image analysis. Since conventional screening processes by cytoscreeners and cytopathologists using microscopes is limited in terms of human resources, it is important to develop new computational techniques that can automatically and rapidly diagnose a large amount of specimens without delay, which would be of great benefit for clinical laboratories and hospitals. The goal of this study was to investigate the use of a deep learning model for the classification of WSIs of LBC specimens into neoplastic and non-neoplastic. To do so, we used a dataset of 1605 cervical WSIs. We evaluated the model on three test sets with a combined total of 1468 WSIs, achieving ROC AUCs for WSI diagnosis in the range of 0.89–0.96, demonstrating the promising potential use of such models for aiding screening processes.

show abstract

“…This operation requires traversing the input from the first time-step to the last one, which is computationally expensive [25]. Even though improved RNN variants such as LSTM [26] and GRU [27] can effectively reduce the difficulty of parameter updates in training, the sequential arrangement of different modal data introduces unnecessary sequential priors, which can force the model to learn an unreasonable one-way information flow while understanding the inter-modal relationships to fit the main features, affecting the effectiveness of feature extraction [28,29].…”

Section: Lite Attention Mechanismmentioning

confidence: 99%

An Attentive Multi-Modal CNN for Brain Tumor Radiogenomic Classification

Xiao

2022

Information

View full text Add to dashboard Cite

Medical images of brain tumors are critical for characterizing the pathology of tumors and early diagnosis. There are multiple modalities for medical images of brain tumors. Fusing the unique features of each modality of the magnetic resonance imaging (MRI) scans can accurately determine the nature of brain tumors. The current genetic analysis approach is time-consuming and requires surgical extraction of brain tissue samples. Accurate classification of multi-modal brain tumor images can speed up the detection process and alleviate patient suffering. Medical image fusion refers to effectively merging the significant information of multiple source images of the same tissue into one image, which will carry abundant information for diagnosis. This paper proposes a novel attentive deep-learning-based classification model that integrates multi-modal feature aggregation, lite attention mechanism, separable embedding, and modal-wise shortcuts for performance improvement. We evaluate our model on the RSNA-MICCAI dataset, a scenario-specific medical image dataset, and demonstrate that the proposed method outperforms the state-of-the-art (SOTA) by around 3%.

show abstract

On the Properties of Neural Machine Translation: Encoder-Decoder Approaches

Cited by 1,072 publications

References 1 publication

Predicting sepsis in multi-site, multi-national intensive care cohorts using deep learning

Predicting sepsis in multi-site, multi-national intensive care cohorts using deep learning

A Deep Learning Model for Cervical Cancer Screening on Liquid-Based Cytology Specimens in Whole Slide Images

An Attentive Multi-Modal CNN for Brain Tumor Radiogenomic Classification

Contact Info

Product

Resources

About