TweetsKB: A Public and Large-Scale RDF Corpus of Annotated Tweets

Abstract

Publicly available social media archives facilitate research in a variety of fields, such as data science, sociology or the digital humanities, where Twitter has emerged as one of the most prominent sources. However, obtaining, archiving and annotating large amounts of tweets is costly. In this paper, we describe TweetsKB, a publicly available corpus of currently more than 1.5 billion tweets, spanning almost 5 years (Jan'13-Nov'17). Metadata information about the tweets as well as extracted entities, hashtags, user mentions and sentiment information are exposed using established RDF/S vocabularies. Next to a description of the extraction and annotation process, we present use cases to illustrate scenarios for entity-centric information exploration, data integration and knowledge discovery facilitated by TweetsKB.

TweetsCOV19 - A Knowledge Base of Semantically Annotated Tweets about the COVID-19 Pandemic

“…The value of a positive (negative) sentiment ranges from +1 (-1) for no positive (no negative) to +5 (-5) for extremely positive (extremely negative). We provide an evaluation of the quality of sentiment annotations produced by SentiStrength over tweets in [10], demonstrating a reasonable performance, in particular in distinguishing stronger sentiments.…”

“…Whereas the processing of TweetsCOV19, described in Section 3, builds on TweetsKB, here, we describe the construction process of TweetsKB as a general, large-scale knowledge base of Twitter discourse. Note that, next to updating the corpus with crawled data after the previous release, improvements were made to the processing pipeline for this release compared to the extraction process described in [10].…”

“…In addition, full-text is exploited for extracting and disambiguating mentioned entities (entity linking), as well as for extracting the magnitude of the expressed positive and negative sentiments (sentiment analysis). Relying on the experimental motivation and prior work in [10], for entity linking, we exploit Yahoo's Fast Entity Linker (FEL) [3]. FEL has shown particularly cost-efficient performance on the task of linking entities from short texts to Wikipedia and is fast and lightweight, being well-suited to run over billions of tweets in a distributed manner.…”

“…We also store the confidence score of each extracted entity to facilitate data consumers to set confidence scores which suit their use cases and requirements when working with our precomputed annotations. The quality of the entity annotations produced by FEL over tweets was evaluated in [10], demonstrating high precision (86%) and an overall satisfactory performance (F1 = 54%). For sentiment analysis, we used SentiStrength [32], a robust and efficient tool for sentiment strength detection on social web data.…”

“…To facilitate a variety of multi-aspect data exploration scenarios, in prior work, we introduced TweetsKB 1 -a pipeline for semantic annotation of tweets and a knowledge base of RDF data for more than 1.5 billion tweets spanning almost five years exposed using established vocabularies [10]. Whereas entity-centric access and exploration methods are crucial to facilitate exploration of large Twitter archives, TweetsKB consistently applies W3C data sharing standards to publish a long-term Twitter archive in the form of an extensible and easy to access knowledge graph including metadata about tweets, user mentions, disambiguated entities, and sentiments.…”