Using Human Attention to Extract Keyphrase from Microblog Post

Zhang, Yingyi; Zhang, Chengzhi

doi:10.18653/v1/p19-1588

Cited by 9 publications

(3 citation statements)

References 14 publications

(24 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Apart from feature fusion, the attention mechanism has been proven as an effective way to further improve the neural model's performance on keyphrase prediction. Yingyi Zhang and Chengzhi Zhang [34] consolidated the attention layer with the RNN model to get a better result than a single RNN.…”

Section: Keyphrase Extractionmentioning

confidence: 99%

“…Sentence Embeddings [29] including more semantic information Pre-trained embedding Embedrank [29] including more semantic information RNN RNN-base model [30] including context information Graph Convolutional Networks GCN DivGraphPointer [31] combination with graph-based method and convolutional network GRU + human attention human attention GRU [34] including context information and focusing on key information BERT + attention AttentionRank [35] Using pre-trained language model, not need to train a model from scratch…”

Section: Keyphrase Extraction Sentence Embeddingmentioning

confidence: 99%

See 1 more Smart Citation

An Enhanced Method on Transformer-Based Model for ONE2SEQ Keyphrase Generation

Shen

2023

Electronics

View full text Add to dashboard Cite

Keyphrase generation is a long-standing task in scientific literature retrieval. The Transformer-based model outperforms other baseline models in this challenge dramatically. In cross-domain keyphrase generation research, topic information plays a guiding role during generation, while in keyphrase generation of individual text, titles can replace topic roles and convey more semantic information. As a result, we proposed an enhanced model architecture named TAtrans. In this research, we investigate the advantages of title attention and sequence code representing phrase order in keyphrase sequence in improving Transformer-based keyphrase generation. We conduct experiments on five widely-used English datasets specifically designed for keyphrase generation. Our method achieves an F1 score in the top five, surpassing the Transformer-based model by 3.2% in KP20k. The results demonstrate that the proposed method outperforms all the previous models on prediction present keyphrases. To evaluate the performance of the proposed model in the Chinese dataset, we construct a new Chinese abstract dataset called CNKIL, which contains a total of 54,546 records. The F1 score of the top five for predicting present keyphrases on the CNKIL dataset exceeds 2.2% compared to the Transformer-based model. However, there is no significant improvement in the model’s performance in predicting absent keyphrases.

show abstract

Section: Keyphrase Extractionmentioning

confidence: 99%

Section: Keyphrase Extraction Sentence Embeddingmentioning

confidence: 99%

An Enhanced Method on Transformer-Based Model for ONE2SEQ Keyphrase Generation

Shen

2023

Electronics

View full text Add to dashboard Cite

show abstract

“…Most previous work in gaze-supported NLP has used gaze as an input feature, e.g. for syntactic sequence labeling [36], classifying referential versus non-referential use of pronouns [82], reference resolution [30], key phrase extraction [86], or prediction of multi-word expressions [64]. Recently, Hollenstein et al [29] proposed to build a lexicon of gaze features given word types, overcoming the need for gaze data at test time.…”

Section: Gaze Integration In Neural Network Architecturesmentioning

confidence: 99%

Improving Natural Language Processing Tasks with Human Gaze-Guided Neural Attention

Sood,

Tannert,

Mueller

et al. 2020

Preprint

View full text Add to dashboard Cite

A lack of corpora has so far limited advances in integrating human gaze data as a supervisory signal in neural attention mechanisms for natural language processing (NLP). We propose a novel hybrid text saliency model (TSM) that, for the first time, combines a cognitive model of reading with explicit human gaze supervision in a single machine learning framework. On four different corpora we demonstrate that our hybrid TSM duration predictions are highly correlated with human gaze ground truth. We further propose a novel joint modeling approach to integrate TSM predictions into the attention layer of a network designed for a specific upstream NLP task without the need for any task-specific human gaze data. We demonstrate that our joint model outperforms the state of the art in paraphrase generation on the Quora Question Pairs corpus by more than 10% in BLEU-4 and achieves state of the art performance for sentence compression on the challenging Google Sentence Compression corpus. As such, our work introduces a practical approach for bridging between data-driven and cognitive models and demonstrates a new way to integrate human gaze-guided neural attention into NLP tasks.34th Conference on Neural Information Processing Systems (NeurIPS 2020),

show abstract

Enhancing keyphrase extraction from microblogs using human reading time

Zhang

2020

Asso for Info Science & Tech

Self Cite

View full text Add to dashboard Cite

The premise of manual keyphrase annotation is to read the corresponding content of an annotated object. Intuitively, when we read, more important words will occupy a longer reading time. Hence, by leveraging human reading time, we can find the salient words in the corresponding content. However, previous studies on keyphrase extraction ignore human reading features. In this article, we aim to leverage human reading time to extract keyphrases from microblog posts. There are two main tasks in this study. One is to determine how to measure the time spent by a human on reading a word. We use eye fixation durations (FDs) extracted from an open source eye‐tracking corpus. Moreover, we propose strategies to make eye FD more effective on keyphrase extraction. The other task is to determine how to integrate human reading time into keyphrase extraction models. We propose two novel neural network models. The first is a model in which the human reading time is used as the ground truth of the attention mechanism. In the second model, we use human reading time as the external feature. Quantitative and qualitative experiments show that our proposed models yield better performance than the baseline models on two microblog datasets.

show abstract

Using Human Attention to Extract Keyphrase from Microblog Post

Cited by 9 publications

References 14 publications

An Enhanced Method on Transformer-Based Model for ONE2SEQ Keyphrase Generation

An Enhanced Method on Transformer-Based Model for ONE2SEQ Keyphrase Generation

Improving Natural Language Processing Tasks with Human Gaze-Guided Neural Attention

Enhancing keyphrase extraction from microblogs using human reading time

Contact Info

Product

Resources

About