See, hear, read: Leveraging multimodality with guided attention for abstractive text summarization

Atri, Yash Kumar; Pramanick, Shraman; Goyal, Vikram; Chakraborty, Tanmoy

doi:10.1016/j.knosys.2021.107152

Cited by 14 publications

(10 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…T P + T N T P + F P + F N + T N (7) The statistic used to assess a model's capacity to forecast true negatives in each accessible category is called specificity, which is expressed mathematically in Equation 8 [35].…”

Section: Accuracy =mentioning

confidence: 99%

See 1 more Smart Citation

Comparison of the Performance of Machine Learning Techniques in the Prediction of Employee

Adeniyi

Adeniyi²,

Oguns

et al. 2022

paradigmplus

View full text Add to dashboard Cite

HR's purpose is to assign the best people to the right job at the right time, train and qualify them, and provide evaluation methods to track their performance and safeguard employees' perspective skills. These data are crucial for decision-makers, but collecting the best and most useful information from such large amounts of data is tough. HR employees no longer need to manually handle vast amounts of data with the advent of data mining. The basic purpose of data mining is to extract information from hidden patterns and trends in data to get near-optimal results. This study aims at comparing the performance of three techniques in the prediction of performance. The dataset undergoes preprocessing steps that include data cleaning, and data compression using PCA. After preprocessing, training and classification were done using Artificial Neural Network, Random Forest, and Decision tree algorithm. The result showed that Artificial Neural networks performed the best for the prediction of employee performance.

show abstract

Section: Accuracy =mentioning

confidence: 99%

“…Of the three types, only the supervised use a pre-trained model for prediction. It creates a model using training data [7,8]. To make the prediction, a training algorithm (machine learning) is used to develop a model.…”

Section: Introductionmentioning

confidence: 99%

Comparison of the Performance of Machine Learning Techniques in the Prediction of Employee

Adeniyi

Adeniyi²,

Oguns

et al. 2022

paradigmplus

View full text Add to dashboard Cite

show abstract

“…The abstract of a paper often serves as a bird's eye view of the paper, highlighting the problem statement, datasets, proposed methodology, analysis, etc. Recent studies [1] re-purpose abstracts to generate summaries of scientific articles. However, it is cumbersome to go through the abstract of each paper.…”

Section: Introductionmentioning

confidence: 99%

“…On the other hand, scientific documents have a complex and structured vocabulary, which the existing methods [42] of generating short summaries are not equipped to handle. Recently, Atri et al [1] proposed as a novel dataset for the multimodal text summarization of scientific presentations; however, it uses the abstract as the target summary, which falls short in producing coherent summaries for the extreme multimodal summarization (TL;DR) task.…”

Section: Introductionmentioning

confidence: 99%

An abstractive text summarization technique using transformer model with self-attention mechanism

Kumar

Solanki

2023

Neural Comput & Applic

View full text Add to dashboard Cite

The realm of scientific text summarization has experienced remarkable progress due to the availability of annotated brief summaries and ample data. However, the utilization of multiple input modalities, such as videos and audio, has yet to be thoroughly explored. At present, scientific multimodal-input-based text summarization systems tend to employ longer target summaries like abstracts, leading to an underwhelming performance in the task of text summarization.In this paper, we deal with a novel task of extreme abstractive text summarization (aka TL;DR generation) by leveraging multiple input modalities. To this end, we introduce mTLDR, a first-of-itskind dataset for the aforementioned task, comprising videos, audio, and text, along with both author-composed summaries and expertannotated summaries. The mTLDR dataset accompanies a total of 4, 182 instances collected from various academic conference proceedings, such as ICLR, ACL, and CVPR. Subsequently, we present mTLDRgen, an encoder-decoder-based model that employs a novel dual-fused hyper-complex Transformer combined with a Wasserstein Riemannian Encoder Transformer, to dexterously capture the intricacies between different modalities in a hyper-complex latent geometric space. The hyper-complex Transformer captures the intrinsic properties between the modalities, while the Wasserstein Riemannian Encoder Transformer captures the latent structure of the modalities in the latent space geometry, thereby enabling the model to produce diverse sentences. mTLDRgen outperforms 20 baselines on mTLDR as well as another non-scientific dataset (How2) across three Rouge-based evaluation measures. Furthermore, based on the qualitative metrics, BERTScore and FEQA, and human evaluations, we demonstrate that the summaries generated by mTLDRgen are fluent and congruent to the original source material. CCS CONCEPTS• >Computing methodologies → Natural language processing.

show abstract

“…The text summarization techniques which is under Natural Language Processing (NLP) are classified into two: extractive and abstractive methods. The extractive method [6][7] [8] concatenates necessary sentences from the original document without modification, while the abstractive process [9][10] [11] retains the original document's meaning but creates new phrases. However, extractive encounters issues in the coherence of the sentences that suffer the sense of the summary; meanwhile, abstractive produces human-like output but tend to compromise the semantic information in summary.…”

Section: Introductionmentioning

confidence: 99%

OCR-based Hybrid Image Text Summarizer using Luhn Algorithm with FinetuneTransformer Modelsfor Long Document

Singco¹,

Trillo²,

Abalorio³

et al. 2023

IJETAE

View full text Add to dashboard Cite

The accessibility of an enormous number of image text documents on the internet has expanded the opportunities to develop a system for image text recognition with text summarization. Several approaches used in ATS in the literature are based on extractive and abstractive techniques; however, few implementations of the hybrid approach were observed. This paper employed state-of-the-art transformer models with the Luhn algorithm for extracted texts using Tesseract OCR. Nine models were generated and tested using the hybrid text summarization approach. Using ROUGE metrics, we compared the proposed system finetune abstractive models against existing abstractive models that use the same dataset Xsum. As a result, the finetune model got the highest ROUGE score during evaluation; in ROUGE-1 score was 57%, the ROUGE-2 score was 43%, and the ROUGE-L score was 42%. Furthermore, even when better algorithms and models were available for summarization, the Luhn algorithm and T5 finetune model provided significant results.

show abstract

See, hear, read: Leveraging multimodality with guided attention for abstractive text summarization

Cited by 14 publications

References 21 publications

Comparison of the Performance of Machine Learning Techniques in the Prediction of Employee

Comparison of the Performance of Machine Learning Techniques in the Prediction of Employee

An abstractive text summarization technique using transformer model with self-attention mechanism

OCR-based Hybrid Image Text Summarizer using Luhn Algorithm with FinetuneTransformer Modelsfor Long Document

Contact Info

Product

Resources

About