ML-Decoder: Scalable and Versatile Classification Head

Ridnik, Tal; Sharir, Gilad; Ben-Cohen, Avi; Ben-Baruch, Emanuel; Noy, Asaf

doi:10.48550/arxiv.2111.12933

Cited by 4 publications

(7 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…When more public ChIP-seq data is available in future, the pre-training model's decoder is able to characterize the dependence among additional TFs. From the technical perspective, we are able to redesign the decoder structure [50] to efficiently scale to thousands of labels (a.k.a. TF binding profiles) without worrying the computational burden.…”

Section: Discussionmentioning

confidence: 99%

A generalizable framework to comprehensively predict epigenome, chromatin organization, and transcriptome

Zhang

Qiu

Feng

et al. 2022

Preprint

View full text Add to dashboard Cite

Many deep learning approaches have been proposed to connect DNA sequence, epigenetic profiles, chromatin organization and transcription activities. While these approaches achieve satisfactory performance in predicting one modality from another, the representations learned are not generalizable across predictive tasks or across cell types. In this paper, we propose a deep learning approach named EPCOT which employs a pre-training and fine-tuning framework, and comprehensively predicts epigenome, chromatin organization, transcriptome, and enhancer activity in one framework, which is also generalizable to new cell types. EPCOT not only achieves superior predictive performance in individual predictive tasks, it also produces globally optimized sequence representations that are generalizable across different predictive tasks. Interpreting EPCOT model also allows us to provide a number of tools and services to the research community including mapping between different genomic modalities, identifying TF sequence binding patterns, and analyzing cell-type specific TF impacts to enhancer activity.

show abstract

Section: Discussionmentioning

confidence: 99%

A generalizable framework to comprehensively predict epigenome, chromatin organization, and transcriptome

Zhang

Qiu

Feng

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…When more public ChIP-seq data is available in future, the pre-training model's decoder is able to characterize the dependence among additional TFs. From the technical perspective, we are able to redesign the decoder structure [49] to efficiently scale to thousands of labels (a.k.a. TF binding profiles) without much computational burden.…”

Section: Discussionmentioning

confidence: 99%

A generalizable framework to comprehensively predict epigenome, chromatin organization, and transcriptome

Zhang

Qiu

Feng

et al. 2022

Preprint

View full text Add to dashboard Cite

Many deep learning approaches have been proposed to connect DNA sequence, epigenetic profiles, chromatin organization, and transcription activity. While these approaches achieve satisfactory performance in predicting one modality from another, the learned representations are not generalizable across predictive tasks or across cell types. In this paper, we propose a deep learning approach named EPCOT which employs a pre-training and fine-tuning framework, and comprehensively predicts epigenome, chromatin organization, transcriptome, and enhancer activity in one framework, which is also generalizable to new cell/tissue types. EPCOT is the first framework proposed to connect all of these genome modalities and achieves superior predictive performance in individual modality prediction. EPCOT also maps from DNA sequence and chromatin accessibility profiles to vectors of generic representations which are generalizable across different modalities and characterize the connections among these modalities. Interpreting EPCOT model also allows us to provide a number of tools and services to the research community including mapping between different genomic modalities, identifying TF sequence binding patterns, and analyzing cell-type specific TF impacts to enhancer activity.

show abstract

“…Having a model trained to predict labels that include nouns, attributes and verbs should provide a set of rich keywords. For this reason, the ML-Decoder [35] model which follows a Transformerbased encoder-decoder pipeline was used. A pre-trained ML-Decoder was adopted to detect the 80 COCO object labels (ML-objs) to provide comparison with the Faster R-CNN object detector results.…”

Section: B Detected Abstractmentioning

confidence: 99%

KENGIC: KEyword-driven and N-Gram Graph based Image Captioning

Birmingham¹,

Muscat²

2023

Preprint

View full text Add to dashboard Cite

This paper presents a Keyword-driven and N-gram Graph based approach for Image Captioning (KENGIC). Most current state-of-the-art image caption generators are trained endto-end on large scale paired image-caption datasets which are very laborious and expensive to collect. Such models are limited in terms of their explainability and their applicability across different domains. To address these limitations, a simple model based on N-Gram graphs which does not require any end-toend training on paired image captions is proposed. Starting with a set of image keywords considered as nodes, the generator is designed to form a directed graph by connecting these nodes through overlapping n-grams as found in a given text corpus. The model then infers the caption by maximising the most probable n-gram sequences from the constructed graph. To analyse the use and choice of keywords in context of this approach, this study analysed the generation of image captions based on (a) keywords extracted from gold standard captions and (b) from automatically detected keywords. Both quantitative and qualitative analyses demonstrated the effectiveness of KENGIC. The performance achieved is very close to that of current state-of-the-art image caption generators that are trained in the unpaired setting. The analysis of this approach could also shed light on the generation process behind current top performing caption generators trained in the paired setting, and in addition, provide insights on the limitations of the current most widely used evaluation metrics in automatic image captioning.

show abstract

ML-Decoder: Scalable and Versatile Classification Head

Cited by 4 publications

References 25 publications

A generalizable framework to comprehensively predict epigenome, chromatin organization, and transcriptome

A generalizable framework to comprehensively predict epigenome, chromatin organization, and transcriptome

A generalizable framework to comprehensively predict epigenome, chromatin organization, and transcriptome

KENGIC: KEyword-driven and N-Gram Graph based Image Captioning

Contact Info

Product

Resources

About