Relevancer: Finding and Labeling Relevant Information in Tweet Collections

Abstract

Abstract. We introduce a tool that supports knowledge workers who want to gain insights from a tweet collection, but due to time constraints cannot go over all tweets. Our system first pre-processes, de-duplicates, and clusters the tweets. The detected clusters are presented to the expert as so-called information threads. Subsequently, based on the information thread labels provided by the expert, a classifier is trained that can be used to classify additional tweets. As a case study, the tool is evaluated on a tweet collection based on the key terms 'genocide' and 'Rohingya'. The average precision and recall of the classifier on six classes is 0.83 and 0.82 respectively. At this level of performance, experts can use the tool to manage tweet collections efficiently without missing much information.

COVCOR20 at WNUT-2020 Task 2: An Attempt to Combine Deep Learning and Expert rules

“…We could extend the training set with cluster mining using Relevancer (Hürriyetoglu et al, 2016), use the rule-based system to extend the training set (Hürriyetoglu, 2019), or use the rule-based system to generate fine-grained data that can be used in a multi-task setting.…”

COVCOR20 at WNUT-2020 Task 2: An Attempt to Combine Deep Learning and Expert rules

“…We could extend the training set with cluster mining using Relevancer (Hürriyetoglu et al, 2016), use the rule-based system to extend the training set (Hürriyetoglu, 2019), or use the rulebased system to generate fine-grained data that can be used in a multi-task setting.…”

Zero-Shot Ranking Socio-Political Texts with Transformer Language Models to Reduce Close Reading Time

“…For the information retrieval process positively labeled documents in a dataset are important and should not be missed, therefore achieving high recall is extremely important. However, there is generally a large number of documents that are relevant or not to the concerned topic and doing close reading for all documents and annotating them requires lots of time and resources (Hürriyetoglu et al, 2016;Hürriyetoǧlu et al, 2017). Therefore, ranking documents according to relevance to the investigated class may help to reduce close reading time and decrease the likelihood of missing critical information.…”