Story Forest

Abstract

Extracting events accurately from vast news corpora and organize events logically is critical for news apps and search engines, which aim to organize news information collected from the Internet and present it to users in the most sensible forms. Intuitively speaking, an event is a group of news documents that report the same news incident possibly in different ways. In this article, we describe our experience of implementing a news content organization system at Tencent to discover events from vast streams of breaking news and to evolve news story structures in an online fashion. Our real-world system faces unique challenges in contrast to previous studies on topic detection and tracking (TDT) and event timeline or graph generation, in that we (1) need to accurately and quickly extract distinguishable events from massive streams of long text documents, and (2) must develop the structures of event stories in an online manner, in order to guarantee a consistent user viewing experience. In solving these challenges, we propose Story Forest , a set of online schemes that automatically clusters streaming documents into events, while connecting related events in growing trees to tell evolving stories. A core novelty of our Story Forest system is EventX , a semi-supervised scheme to extract events from massive Internet news corpora. EventX relies on a two-layered, graph-based clustering procedure to group documents into fine-grained events. We conducted extensive evaluations based on (1) 60 GB of real-world Chinese news data, (2) a large Chinese Internet news dataset that contains 11,748 news articles with truth event labels, and (3) the 20 News Groups English dataset, through detailed pilot user experience studies. The results demonstrate the superior capabilities of Story Forest to accurately identify events and organize news text into a logical structure that is appealing to human readers.

Continuous Similarity Learning with Shared Neural Semantic Representation for Joint Event Detection and Evolution

“…Following the original Topic Detection and Tracking (TDT) program [ 10 ], event detection aims to discover new or previously unidentified events from the information stream. Most of the works detect events of interest from news media [ 8 , 11 , 19 ] or social media [ 5 , 20 , 21 ]. In addition, some research works are also called event detection but are essentially different from the former task; for example, [ 22 ] talks about ACE event detection task which focuses on extracting events with entities from sentences and [ 23 ] considers the event detection as a text classification problem, that is, categorizing each event to predefined types.…”

“…However, they do not deal with newly generated documents at any time and update story structures in an online manner. Recently, [ 8 ] proposes a structure of story tree to characterize the evolution of the story, grows story trees with events online, and presents the most effective representation effect according to extensive user experience. In this paper, various evolution structures are investigated in conjunction with deep learning based event clustering algorithm.…”

“…An event E is a particular thing that happens at a specific time and place [ 7 , 43 ]. Specifically, it is a set of news documents reporting the same real-world breaking news [ 8 ].…”

“…Generally, an event refers to a particular thing that happens at a specific time and place [ 7 ], for example, Opening Ceremony of the 29 th Modern Summer Olympic Games held in Beijing on August 8 th , 2008. Under the premise of this definition, event detection aims to identify a group of news documents that report the same topic possibly in different ways [ 8 ]. Therefore, a new topic can be detected if new news documents are found not belonging to existing topic clusters.…”

“…For decades, research has introduced many methods for event detection [ 5 , 6 , 9 ] and event evolution [ 1 – 4 , 8 ] in the area of Topic Detection and Tracking (TDT) [ 10 ]. Of these methods, a similar pipeline is followed; that is, event detection and evolution are considered as two independent subtasks.…”