Multilingual Clustering of Streaming News

Abstract

Clustering news across languages enables efficient media monitoring by aggregating articles from multilingual sources into coherent stories. Doing so in an online setting allows scalable processing of massive news streams. To this end, we describe a novel method for clustering an incoming stream of multilingual documents into monolingual and crosslingual story clusters. Unlike typical clustering approaches that consider a small and known number of labels, we tackle the problem of discovering an ever growing number of cluster labels in an online fashion, using real news datasets in multiple languages. Our method is simple to implement, computationally efficient and produces state-of-the-art results on datasets in German, English and Spanish.

Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing: Tutorial Abstracts

“…The task of identifying and tracking events was first introduced in the Topic Detection and Tracking challenge (Allan et al, 1998). Recent work has explored new methods for tracking and visualizing such events over time (e.g., (Laban and Hearst, 2017;Miranda et al, 2018;Staykovski et al, 2019;Saravanakumar et al, 2021)), in some cases generating summaries that contain information on what is new (e.g., (Kedzie et al, 2015(Kedzie et al, , 2018) and in other cases, exploring timeline summarization, ordering events and generating summaries that are placed along a timeline (e.g., (Wang et al, 2015;Binh Tran et al, 2013;Nguyen et al, 2014)) We will also consider how these are related to summarization of an event that takes place within a single day, a problem that falls within the category of multidocument summarization (e.g., (Liu and Lapata, 2019;Fabbri et al, 2019)), as typically there may be many articles covering the same event. By using multiple articles as input, a summarizer can present different perspectives on the same event as well as identify salient information that is highlighted many in different ways across the set of input articles.…”

Tanbih: Get To Know What You Are Reading

“…The model has state-of-the-art performance, on the test partition of the corpus from Miranda et al (2018): F 1 = 98.11 and F 1 BCubed = 94.41 (F 1 BCubed is an evaluation measure specifically designed to evaluate clustering algorithms (Amigó et al, 2009)). As a comparison, the best model in (Miranda et al, 2018) has F 1 = 94.1 (see Staykovski et al (2019) for further details). In Figure 2 the article from the Sputnik is flagged as likely to be propagandistic by our system.…”

“…We selected all parameters empirically on the training part of the corpus from (Miranda et al, 2018). The sequence of overlapping local graphs is merged in the order of their creation, thus generating stories from the topics.…”

“…For step (ii), the topics created from the preceding stage are merged if the cosine similarity sim(t i , t j ) ≥ T 2 , where t i (t j ) is the mean of all vectors belonging to topic i (j), with T 2 = 0.8. The model achieved state-of-the-art performance on the testing partition of the corpus from Miranda et al (2018): an F 1 of 98.11 and an F 1 BCubed of 94.41. 15 As a comparison, the best model described in (Miranda et al, 2018) achieved an F 1 of 94.1.…”

“…The model achieved state-of-the-art performance on the testing partition of the corpus from Miranda et al (2018): an F 1 of 98.11 and an F 1 BCubed of 94.41. 15 As a comparison, the best model described in (Miranda et al, 2018) achieved an F 1 of 94.1. See Staykovski et al (2019) for further details.…”