Automatic Title Generation for Text with Pre-trained Transformer Language Model

Mishra, Prakhar; Diwan, Chaitali; Srinivasa, Srinath; Srinivasaraghavan, G.

doi:10.1109/icsc50631.2021.00009

Cited by 12 publications

(6 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Mishra et al [10] proposed a paper title generation method using GPT-2, which is a transformer-based natural language processing model similar to BERT. Because GPT-2 has a probabilistic characteristic, its output varies each time.…”

Section: Title Generation Methods Using Dnnmentioning

confidence: 99%

Development of an Extractive Title Generation System Using Titles of Papers of Top Conferences for Intermediate English Students

Kaku,

Kikuchi,

Ozono

et al. 2021

Preprint

View full text Add to dashboard Cite

The formulation of good academic paper titles in English is challenging for intermediate English authors (particularly students). This is because such authors are not aware of the type of titles that are generally in use. We aim to realize a support system for formulating more effective English titles for intermediate English and beginner authors. This study develops an extractive title generation system that formulates titles from keywords extracted from an abstract. Moreover, we realize a title evaluation model that can evaluate the appropriateness of paper titles. We train the model with titles of top-conference papers by using BERT. This paper describes the training data, implementation, and experimental results. The results show that our evaluation model can identify top-conference titles more effectively than intermediate English and beginner students.

show abstract

Section: Title Generation Methods Using Dnnmentioning

confidence: 99%

Development of an Extractive Title Generation System Using Titles of Papers of Top Conferences for Intermediate English Students

Kaku,

Kikuchi,

Ozono

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Title generation and evaluation Mishra et al (2021) perform A2T with pre-trained GPT-2 finetuned on arxiv papers and subsequent (rule-based) modules of title selection and refinement. We compare many more text generation models for the task, use better evaluation (including more comprehensive human and automatic evaluation), do not make use of rule-based selection and also consider humor in title generation.…”

Section: Related Workmentioning

confidence: 99%

“…We also leverage the relationship to summarization by considering pre-trained models fine-tuned on summarization datasets. In contrast to Putra and Khodra (2017) and Mishra et al (2021), we only consider end-to-end models that do not involve pipelines. While refinement steps could be further helpful (but also error-prone), they additionally require potentially undesirable human intervention (Belouadi and Eger, 2023).…”

Section: Related Workmentioning

confidence: 99%

Transformers Go for the LOLs: Generating (Humourous) Titles from Scientific Abstracts End-to-End

Chen,

Eger

2023

Proceedings of the 4th Workshop on Evaluation and Comparison of NLP Systems

View full text Add to dashboard Cite

We consider the end-to-end abstract-to-title generation problem, exploring seven recent transformer based models (including ChatGPT) fine-tuned on more than 30k abstract-title pairs from NLP and machine learning (ML) venues. As an extension, we also consider the harder problem of generating humorous paper titles. For the latter, we compile the first large-scale humor annotated dataset for scientific papers in the NLP/ML domains, comprising ∼2.6k titles. We evaluate all models using human and automatic metrics. Our human evaluation suggests that our best end-to-end system performs similarly to human authors (but arguably slightly worse). Generating funny titles is more difficult, however, and our automatic systems clearly underperform relative to humans and often learn dataset artefacts of humor. Finally, ChatGPT, without any fine-tuning, performs on the level of our best fine-tuned system. 1

show abstract

“…For strictly extractive endeavors, decoders are typically substituted by a specific classifier determining which input tokens will appear in the final summary. Another strategy is to fine-tune a GPT-2 (Radford et al, 2019) style auto-regressive model for the summarization task; this approach was adopted by both Koppatz et al (2022) for headline generation and Mishra et al (2021) for title generation. Many contemporary title and headline generation methods have adopted metrics like BLEU or ROUGE to assess model performance (Matsumaru et al, 2020;Bukhtiyarov and Gusev, 2020;Tilk and Alumäe, 2017;Mishra et al, 2021); these are also standard for summarization evaluation.…”

Section: Related Workmentioning

confidence: 99%

“…Another strategy is to fine-tune a GPT-2 (Radford et al, 2019) style auto-regressive model for the summarization task; this approach was adopted by both Koppatz et al (2022) for headline generation and Mishra et al (2021) for title generation. Many contemporary title and headline generation methods have adopted metrics like BLEU or ROUGE to assess model performance (Matsumaru et al, 2020;Bukhtiyarov and Gusev, 2020;Tilk and Alumäe, 2017;Mishra et al, 2021); these are also standard for summarization evaluation. An exception is Koppatz et al (2022), who also rely on manual structured review by domain experts to assess the quality of their automatically generated headlines.…”

Section: Related Workmentioning

confidence: 99%

Enhancing Academic Title Generation Using SciBERT and Linguistic Rules

Callegari,

Vajdecka,

Xhura

et al. 2023

Proceedings of the Second Workshop on Information Extraction From Scientific Publications

View full text Add to dashboard Cite

This study tackles the challenge of generating appropriate academic titles based on the paper's abstract. We approach this task as a high-level text summarization problem and introduce an innovative post-processing method that combines a predictive model with a set of linguistic rules to enhance the quality of the title generation. We start by evaluating three Natural Language Generation models (BART, T5, Flan T5), by identifying the top-performing model and by configuring it to generate diverse titles. We then conduct experiments employing various post-processing strategies -using SciBERT and linguistic rules-to select the best title out of all machine-generated options. Finally, we assess our title selection methods in relation to human evaluations.

show abstract

Automatic Title Generation for Text with Pre-trained Transformer Language Model

Cited by 12 publications

References 12 publications

Development of an Extractive Title Generation System Using Titles of Papers of Top Conferences for Intermediate English Students

Development of an Extractive Title Generation System Using Titles of Papers of Top Conferences for Intermediate English Students

Transformers Go for the LOLs: Generating (Humourous) Titles from Scientific Abstracts End-to-End

Enhancing Academic Title Generation Using SciBERT and Linguistic Rules

Contact Info

Product

Resources

About