Automatic generation of overview timelines

Abstract

We present a statistical model of feature occurrence over time, and develop tests based on classical hypothesis testing for significance of term appearance on a given date. Using additional classical hypothesis testing we are able to combine these terms to generate "topics" as defined by the Topic Detection and Tracking study. The groupings of terms obtained can be used to automatically generate an interactive timeline displaying the major events and topics covered by the corpus. To test the validity of our technique we extracted a large number of these topics from a test corpus and had human evaluators judge how well the selected features captured the gist of the topics, and how they overlapped with a set of known topics from the corpus. The resulting topics were highly rated by evaluators who compared them to known topics.

newsLens: building and visualizing long-ranging news stories

“…In the context of news, the topics detected and tracked are commonly called stories. Swan and Allan (2000) use the Topic Detection and Tracking (TDT) and TDT2 datasets, consisting of 50,000 news articles to produce 146 stories, called clusters. The clustering process is done using named entities and noun phrases, as opposed to unigrams.…”

A Framework for Text Processing and Supporting Access to Collections of Digitized Historical Newspapers

“…Specifically, timelines which display sets of temporal events would be the central organizing metaphor. This could be sets of articles [31] or it could be attributes of Entities such as events in a person's life [2][4] [5]. Figure 7 shows a novel timeline developed and applied to Brooklyn history.…”

Detecting and Browsing Events in Unstructured text

“…For the mid-1990s events in the second TDT study, systems had trouble treating the O. J. Simpson case or the investigation of the Oklahoma city bombing as a single event [17,19]. Many historical documents discuss long-running events -e.g., wars in addition to battles -, and many users will wish to browse digital libraries at a scale larger than events of a few days' length.…”