A Case Study on Neural Headline Generation for Editing Support

Murao, Kazuma; Kobayashi, Kurima; Kobayashi, Hayato; Yatsuka, Taichi; Masuyama, Tohru; Higurashi, Tatsuru; Tabuchi, Yoshimune

doi:10.18653/v1/n19-2010

Cited by 18 publications

(8 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Extractive methods select a subset of actual sentences in original article, which may derive incoherent summary (Alfonseca et al, 2013). While abstractive models, basically falling in an encoderdecoder (Shen et al, 2017a;Murao et al, 2019) framework, can generate more condensed output based on the latent representation of news content. However, the nature of text summarization methods without considering interactions between news and users renders them ineffective in our personalized headline generation.…”

Section: Related Workmentioning

confidence: 99%

“…News headline generation (Dorr et al, 2003;Lopyrev, 2015;Alfonseca et al, 2013;Tan et al, 2017;See et al, 2017;Xu et al, 2019;Murao et al, 2019;Gavrilov et al, 2019;Gu et al, 2020;Song et al, 2020), conventionally considered as a paradigm of challenging text summarization task, has been extensively explored for decades. Their intuitive intention is to empower the model to output a condensed generalization, e.g., one sentence, of a news article.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

PENS: A Dataset and Generic Framework for Personalized News Headline Generation

Ao¹,

Wang²,

Luo³

et al. 2021

Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Confer

View full text Add to dashboard Cite

In this paper, we formulate the personalized news headline generation problem whose goal is to output a user-specific title based on both a user's reading interests and a candidate news body to be exposed to her. To build up a benchmark for this problem, we publicize a large-scale dataset named PENS (PErsonalized News headlineS). The training set is collected from user impressions logs of Microsoft News, and the test set is manually created by hundreds of native speakers to enable a fair testbed for evaluating models in an offline mode. We propose a generic framework as a preparatory solution to our problem. At its heart, user preference is learned by leveraging the user behavioral data, and three kinds of user preference injections are proposed to personalize a text generator and establish personalized headlines. We investigate our dataset by implementing several state-of-the-art user modeling methods in our framework to demonstrate a benchmark score for the proposed dataset. The dataset is available at https: //msnews.github.io/pens.html.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

PENS: A Dataset and Generic Framework for Personalized News Headline Generation

Ao¹,

Wang²,

Luo³

et al. 2021

Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Confer

View full text Add to dashboard Cite

show abstract

“…In the real world, most written articles already have a corresponding headline, which makes those models less useful. More recent work (Murao et al 2019) proposes a new headline editing task, in which the goal is to create a shorter headline from the original headline. However, many writers want a more powerful tool than just making the title shorter.…”

Section: Related Work Headline Editingmentioning

confidence: 99%

“…In most cases, we can never have enough training data. Many existing studies on headline generation (Murao et al 2019;Ayana et al 2017) are data-hungry and cannot generate satisfactory results in real applications. Low resource also leads to text degeneration, such as repetition.…”

Section: Introductionmentioning

confidence: 99%

Importance-Aware Learning for Neural Headline Editing

Li²,

Zhou³

et al. 2020

AAAI

View full text Add to dashboard Cite

Many social media news writers are not professionally trained. Therefore, social media platforms have to hire professional editors to adjust amateur headlines to attract more readers. We propose to automate this headline editing process through neural network models to provide more immediate writing support for these social media news writers. To train such a neural headline editing model, we collected a dataset which contains articles with original headlines and professionally edited headlines. However, it is expensive to collect a large number of professionally edited headlines. To solve this low-resource problem, we design an encoder-decoder model which leverages large scale pre-trained language models. We further improve the pre-trained model's quality by introducing a headline generation task as an intermediate task before the headline editing task. Also, we propose Self Importance-Aware (SIA) loss to address the different levels of editing in the dataset by down-weighting the importance of easily classified tokens and sentences. With the help of Pre-training, Adaptation, and SIA, the model learns to generate headlines in the professional editor's style. Experimental results show that our method significantly improves the quality of headline editing comparing against previous methods.

show abstract

“…Over the past decade, both rule-based [9], compression-based [13,14] and statistical-based methods [1,65] have been explored with handcrafted features and linguistic rules. Recent state-of-the-art headline generation models are dominated by end-to-end encoder-decoder architectures [16,19,36,53,68]. Similar to summarization models, the encoder module considers different formats of input representation including word position embedding [7], abstractive meaning representations [58] and other linguistic features [38].…”

Section: Related Workmentioning

confidence: 99%

Generating Representative Headlines for News Stories

Gu¹,

Mao²,

Liu³

et al. 2020

Preprint

View full text Add to dashboard Cite

Millions of news articles are published online every day, which can be overwhelming for readers to follow. Grouping articles that are reporting the same event into news stories is a common way of assisting readers in their news consumption. However, it remains a challenging research problem to efficiently and effectively generate a representative headline for each story. Automatic summarization of a document set has been studied for decades, while few studies have focused on generating representative headlines for a set of articles. Unlike summaries, which aim to capture most information with least redundancy, headlines aim to capture information jointly shared by the story articles in short length and exclude information specific to each individual article.In this work, we study the problem of generating representative headlines for news stories. We develop a distant supervision approach to train large-scale generation models without any human annotation. The proposed approach centers on two technical components. First, we propose a multi-level pre-training framework that incorporates massive unlabeled corpus with different qualityvs.-quantity balance at different levels. We show that models trained within the multi-level pre-training framework outperform those only trained with human-curated corpus. Second, we propose a novel self-voting-based article attention layer to extract salient information shared by multiple articles. We show that models that incorporate this attention layer are robust to potential noises in news stories and outperform existing baselines on both clean and noisy datasets. We further enhance our model by incorporating human labels, and show that our distant supervision approach significantly reduces the demand on labeled data. Finally, to serve the research community, we publish the first manually curated benchmark dataset on headline generation for news stories, NewSHead, which contains 367K stories (each with 3-5 articles), 6.5 times larger than the current largest multi-document summarization dataset.2 https://www.blog.google/products/news/new-google-news-ai-meets-human-intelligence/

show abstract

A Case Study on Neural Headline Generation for Editing Support

Cited by 18 publications

References 32 publications

PENS: A Dataset and Generic Framework for Personalized News Headline Generation

PENS: A Dataset and Generic Framework for Personalized News Headline Generation

Importance-Aware Learning for Neural Headline Editing

Generating Representative Headlines for News Stories

Contact Info

Product

Resources

About