IITK at SemEval-2020 Task 10: Transformers for Emphasis Selection

Singhal, Vipul; Dhull, Sahil; Agarwal, Rishabh; Modi, Ashutosh

doi:10.48550/arxiv.2007.10820

Cited by 2 publications

(8 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Figure 4 presents the Emphasis Heatmap for some examples from the test set using our final model. In Table 4 we benchmark our EmpLite model with the state-of-the-art solution by IITK (Singhal et al, 2020) which utilized huge pretrained models like ELMo, BERT (Devlin et al, 2018), RoBERTa (Liu et al, 2019) and XLNet (Yang et al, 2019). These models require huge RAM/ROM for on-device inferencing making it unsuitable for edge devices where resources are constrained.…”

Section: Experimental Settings and Resultsmentioning

confidence: 99%

“…Pre-trained language model has also been used to achieve emphasis selection (Huang et al, 2020). Singhal et al (Singhal et al, 2020) achieves significantly good performance with (a) Bi-LSTM + Attention approach, and (b) Transformers approach. To achieve their modest performances, these architectures produce huge models.…”

Section: Introductionmentioning

confidence: 99%

“…To achieve their modest performances, these architectures produce huge models. For instance, IITK model (Singhal et al, 2020) takes up 469.20 MB in BiLSTM + Attention approach, while requiring almost 1.5 GB in Transformers approach. This is partly due to the use of embeddings like BERT (1.2 GB) (Devlin et al, 2018), XLNET (1.34 GB) (Yang et al, 2019), RoBERTa (1.3 GB) (Liu et al, 2019), etc.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

EmpLite: A Lightweight Sequence Labeling Model for Emphasis Selection of Short Texts

Agarwal¹,

Ghosh²,

Chalamalasetti³

et al. 2021

Preprint

View full text Add to dashboard Cite

Word emphasis in textual content aims at conveying the desired intention by changing the size, color, typeface, style (bold, italic, etc.), and other typographical features. The emphasized words are extremely helpful in drawing the readers' attention to specific information that the authors wish to emphasize. However, performing such emphasis using a soft keyboard for social media interactions is time-consuming and has an associated learning curve. In this paper, we propose a novel approach to automate the emphasis word detection on short written texts. To the best of our knowledge, this work presents the first lightweight deep learning approach for smartphone deployment of emphasis selection. Experimental results show that our approach achieves comparable accuracy at a much lower model size than existing models. Our best lightweight model has a memory footprint of 2.82 MB with a matching score of 0.716 on SemEval-2020 (shallowLearner, 2020) public benchmark dataset.

show abstract

Section: Experimental Settings and Resultsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

EmpLite: A Lightweight Sequence Labeling Model for Emphasis Selection of Short Texts

Agarwal¹,

Ghosh²,

Chalamalasetti³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…In our transformers approach, we experiment with two different transformer based model architectures, namely RoBERTa (Liu et al 2019) and XLNet (Yang et al 2019). Our choice of transformer architectures is inspired by the best performing architectures in SemEval-2020 Task 10, Emphasis selection for written text in visual media (Singhal et al 2020;Anand et al 2020). Both these models were pre-trained on large amounts of unannotated data in an unsupervised manner.…”

Section: Introductionmentioning

confidence: 99%

“…Additionally, we also concatenate some word-level features in the attention output before feeding it to the fully connected layers. A part of our modification is inspired by the team that stood 3rd on the SemEval-2020 Task 10 leaderboard (Singhal et al 2020).…”

Section: Introductionmentioning

confidence: 99%

Cisco at AAAI-CAD21 shared task: Predicting Emphasis in Presentation Slides using Contextualized Embeddings

Ghosh,

Kumar,

Jalan

et al. 2021

Preprint

View full text Add to dashboard Cite

This paper describes our proposed system for the AAAI-CAD21 shared task: Predicting Emphasis in Presentation Slides. In this specific task, given the contents of a slide we are asked to predict the degree of emphasis to be laid on each word in the slide. We propose 2 approaches to this problem including a BiLSTM-ELMo approach and a transformers based approach based on RoBERTa and XLNet architectures. We achieve a score of 0.518 on the evaluation leaderboard which ranks us 3 rd and 0.543 on the post-evaluation leaderboard which ranks us 1 st at the time of writing the paper.

show abstract

IITK at SemEval-2020 Task 10: Transformers for Emphasis Selection

Cited by 2 publications

References 0 publications

EmpLite: A Lightweight Sequence Labeling Model for Emphasis Selection of Short Texts

EmpLite: A Lightweight Sequence Labeling Model for Emphasis Selection of Short Texts

Cisco at AAAI-CAD21 shared task: Predicting Emphasis in Presentation Slides using Contextualized Embeddings

Contact Info

Product

Resources

About