Automatic label generation for news comment clusters

Aker, Ahmet; Paramita, Monica Lestari; Kurtić, Emina; Funk, Adam; Barker, Emma; Hepple, Mark; Gaizauskas, Robert

doi:10.18653/v1/w16-6610

Cited by 15 publications

(6 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Over the last decade, some research has been directed at the summarization of forum threads Hovy 2005, 2006;Tigelaar, op den Akker, and Hiemstra 2010). The majority of the recent work addressed summarization of comment threads on news websites (Ren et al 2011;Llewellyn, Grover, and Oberlander 2014;Giannakopoulos et al 2015;Kabadjov et al 2015;Aker et al 2016;Barker et al 2016). In the past decade, abstractive summarization techniques have been successfully applied to tasks related to discussion thread summarization, such as summarizing email threads (Zajic, Dorr, and Lin 2008), summarizing spoken and written conversations and meetings Oya et al 2014), and Twitter topic summarization (Zhang et al 2013).…”

Section: Summarization For Discussion Forum Threadsmentioning

confidence: 99%

Query-based summarization of discussion threads

et al. 2019

View full text Add to dashboard Cite

In this paper, we address query-based summarization of discussion threads. New users can profit from the information shared in the forum, Please check if the inserted city and country names in the affiliations are correct. if they can find back the previously posted information. However, discussion threads on a single topic can easily comprise dozens or hundreds of individual posts. Our aim is to summarize forum threads given real web search queries. We created a data set with search queries from a discussion forum’s search engine log and the discussion threads that were clicked by the user who entered the query. For 120 thread–query combinations, a reference summary was made by five different human raters. We compared two methods for automatic summarization of the threads: a query-independent method based on post features, and Maximum Marginal Relevance (MMR), a method that takes the query into account. We also compared four different word embeddings representations as alternative for standard word vectors in extractive summarization. We find (1) that the agreement between human summarizers does not improve when a query is provided that: (2) the query-independent post features as well as a centroid-based baseline outperform MMR by a large margin; (3) combining the post features with query similarity gives a small improvement over the use of post features alone; and (4) for the word embeddings, a match in domain appears to be more important than corpus size and dimensionality. However, the differences between the models were not reflected by differences in quality of the summaries created with help of these models. We conclude that query-based summarization with web queries is challenging because the queries are short, and a click on a result is not a direct indicator for the relevance of the result.

show abstract

Section: Summarization For Discussion Forum Threadsmentioning

confidence: 99%

Query-based summarization of discussion threads

et al. 2019

View full text Add to dashboard Cite

show abstract

“…Rather than restricting our analysis to the system's output, truly assessing whether an NLG system is producing relevant personalized output necessitates extrinsic, task-based evaluations involving users. Such methods have long been used in the evaluation of automated summarization systems (Hand, 1997;He et al, 1999;Mani, 2001;McKeown et al, 2005) as well as NLG systems (Mellish and Dale, 1998;Reiter et al, 2001;Colineau et al, 2002), though we note that recent years have seen somewhat less of this sort of ecologically valid evaluation, and much more focus on statistical evaluation; the work of Barker et al (2016) and Newman et al (2020) represent examples of very welcome exceptions to this trend.…”

Section: Architectures For Personalizationmentioning

confidence: 91%

Refocusing on Relevance: Personalization in NLG

Dudy¹,

Bedrick²,

Webber³

2021

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing

View full text Add to dashboard Cite

Many NLG tasks such as summarization, dialogue response, or open domain question answering focus primarily on a source text in order to generate a target response. This standard approach falls short, however, when a user's intent or context of work is not easily recoverable based solely on that source texta scenario that we argue is more of the rule than the exception. In this work, we argue that NLG systems in general should place a much higher level of emphasis on making use of additional context, and suggest that relevance (as used in Information Retrieval) be thought of as a crucial tool for designing user-oriented text-generating tasks. We further discuss possible harms and hazards around such personalization, and argue that value-sensitive design represents a crucial path forward through these challenges.

show abstract

“…e goal of DTS is to select the most relevant reply posts in a discussion thread and merge them to form a concise thread summary. Most of the current work focused on comment threads summarization on news websites [48][49][50][51][52][53]. Previous research studies have effectively attempted abstractive summarization (AS) techniques for the task of DTS, such as e-mail threads summarization [3], summarizing written and spoken chats/conversations and meeting recordings [4], and summarizing Twitter topic [5].…”

Section: Related Workmentioning

confidence: 99%

Sentence Embedding Based Semantic Clustering Approach for Discussion Thread Summarization

et al. 2020

View full text Add to dashboard Cite

Huge data on the web come from discussion forums, which contain millions of threads. Discussion threads are a valuable source of knowledge for Internet users, as they have information about numerous topics. The discussion thread related to single topic comprises a huge number of reply posts, which makes it hard for the forum users to scan all the replies and determine the most relevant replies in the thread. At the same time, it is also hard for the forum users to manually summarize the bulk of reply posts in order to get the gist of discussion thread. Thus, automatically extracting the most relevant replies from discussion thread and combining them to form a summary are a challenging task. With this motivation behind, this study has proposed a sentence embedding based clustering approach for discussion thread summarization. The proposed approach works in the following fashion: At first, word2vec model is employed to represent reply sentences in the discussion thread through sentence embeddings/sentence vectors. Next, K-medoid clustering algorithm is applied to group semantically similar reply sentences in order to reduce the overlapping reply sentences. Finally, different quality text features are utilized to rank the reply sentences in different clusters, and then the high-ranked reply sentences are picked out from all clusters to form the thread summary. Two standard forum datasets are used to assess the effectiveness of the suggested approach. Empirical results confirm that the proposed sentence based clustering approach performed superior in comparison to other summarization methods in the context of mean precision, recall, and F-measure.

show abstract

Automatic label generation for news comment clusters

Cited by 15 publications

References 12 publications

Query-based summarization of discussion threads

Query-based summarization of discussion threads

Refocusing on Relevance: Personalization in NLG

Sentence Embedding Based Semantic Clustering Approach for Discussion Thread Summarization

Contact Info

Product

Resources

About