News Monitor: A Framework for Exploring News in Real-Time

Abstract

News articles generated by online media are a major source of information. In this work, we present News Monitor, a framework that automatically collects news articles from a wide variety of online news portals and performs various analysis tasks. The framework initially identifies fresh news (first stories) and clusters articles about the same incidents. For every story, at first, it extracts all of the corresponding triples and, then, it creates a knowledge base (KB) using open information extraction techniques. This knowledge base is then used to create a summary for the user. News Monitor allows for the users to use it as a search engine, ask their questions in their natural language and receive answers that have been created by the state-of-the-art framework BERT. In addition, News Monitor crawls the Twitter stream using a dynamic set of “trending” keywords in order to retrieve all messages relevant to the news. The framework is distributed, online and performs analysis in real-time. According to the evaluation results, the fake news detection techniques utilized by News Monitor allow for a F-measure of 82% in the rumor identification task and an accuracy of 92% in the stance detection tasks. The major contribution of this work can be summarized as a novel real-time and scalable architecture that combines various effective techniques under a news analysis framework.

Exploring Environmental, Social, and Governance (ESG) Discourse in News: An AI-Powered Investigation Through Knowledge Graph Analysis

“…News analysis and monitoring is a broad research area that encompasses various tasks, including first story detection, clustering, trends detection, event detection, question/answering, summarization, and fake news detection [14]- [18].…”

“…The comparison is performed by using the state-of-the-art framework Sentence-Transformers 16 and encoding the entities with the all-mpnetbase-v2 17 transformer model. We chose this model since it showed state-of-the-art performances on a multitude of tasks including semantic text similarity 18 . Entities with a cosine similarity equal to or greater than a threshold eth merge = 0.9 (empirically calculated) are merged.…”

“…SentenceTransformers -https://huggingface.co/sentence-transformers17 all-mpnet-base-v2 -https://huggingface.co/sentence-transformers/ all-mpnet-base-v218 SentenceBERT -https://www.sbert.net/docs/pretrained_models.html6 VOLUME 11, 2023 This article has been accepted for publication in IEEE Access. This is the author's version which has not been fully edited and content may change prior to final publication.…”