NewsCompare - A novel application for detecting news influence in a country

Abstract

The concept of "fake news" has been referenced and thrown around in news reports so much in recent years that it has become a news topic in its own right. At its core, it poses a chilling question -what do we do if our worldview is fundamentally wrong? Even if internally consistent, what if it does not match the real world? Are our beliefs justified, or could we become indoctrinated from living in a "bubble"? If the latter is true, how could we even test the limits of said bubble from within its confines?Without delving into advanced epistemology (i.e. fundamentally "how do we know that we know?"), one of the obvious key requirements is avoiding a reliance on a single source of information. This is no easy task, considering the relative scarcity of truly independent news publishing companies, compared to say, massive conglomerates pushing a certain agenda under different publication names, or (possibly worse) state-sponsored propaganda. Fact-checkers going through articles individually can only do so much, before buckling under the never-ending deluge of information coming out, ranging from nearly-correct to completely fabricated.We propose a new method to augment this process, by speeding up and automating the more cumbersome and time-consuming tasks involved. Our application, NewsCompare takes any list of target websites as input (news-related in our use case, but otherwise not restricted), visits them in parallel and retrieves any text content found within. Web pages are subsequently compared to each other, and similarities are tentatively pointed out. These results can be manually verified in order to determine which websites tend to draw inspiration from one another. The data gathered on every intermediate step can be queried and analyzed separately, and most notably we already use the set of hyperlinks to and from the various websites we encounter to paint a sort of "map" of that particular slice of the web. This map can then be cross-referenced and further strengthen the conclusion that a particular grouping of sites with strong links to each other, and posting similar content, are likely to share the same allegiance.We run our application on the Romanian news websites and we draw several interesting observations.

Detecting News Influence in a Country: One Step Forward Towards Understanding Fake News

“…Exclusively online news outlets on the other hand do not have the luxury of print to fall back on, so we expect them to make that much more of an effort in establishing a foothold in the online market to draw revenue from. This is actually supported by some of our findings, see [37] for a specific example.…”

“…However, we also expect (and welcome) any constructive criticism and bug reports that get us closer to a flawless product. Despite not coming from a sociology background, we try our hand at interpreting the results we get from our use case, at least to the extent that we are aware of what characteristics to look for (see [37] for more details).…”

“…Having readily available access to basic graph details, like node degree and connectivity allows us to see how our results line up with existing research, and potentially put it to the test. See [37] for some actual examples of insight derived from our particular set of target websites.…”