Imagine you’re putting together a publication. You’ve spent countless hours pouring over research literature as you connected the dots leading up to your latest discovery. Knowing that the references you choose to cite create the support for the work you’ve done, you want to be confident that these studies are reliable.

Now imagine you’re an editor or a reviewer. You find time in the middle of an already busy day to review a paper that was recently submitted. Beyond critically reading the draft and giving the authors meaningful feedback, you want to protect the integrity of your journal and make sure someone doesn’t submit a paper that cites previously retracted work. Or maybe even heavily disputed work without proper acknowledgment of the debate.

On both sides of the coin, and to the broader scientific institution, the quality of references is paramount. What if you could check the reliability of references at the click of a button?

Today, we’re happy to introduce a new feature we’ve been working on at scite: Reference Check.

With this, researchers, editors, peer-reviewers, and anyone interested can upload a PDF of a manuscript and immediately see how its references have been cited elsewhere. You can also see if articles were retracted, received an expression of concern, or other editorial notices.

Try it now: scite.ai/#reference-check

For example, looking at the Scientific Reports article “Eosinophils support adipocyte maturation and promote glucose tolerance in obesity” by Lee et. al, you can see a reference with and without the reference check. On the left side is the typical bibliographical information that makes up the reference. On the right side is the same information supplemented by the citation count broken down into supporting, disputing, and mentioning cites, as well as a notice that this reference has been retracted. The text above the reference check shows how it was used in the manuscript, allowing authors, editors, and others to quickly verify that the reference is sound and that it is being used correctly.

Identifying reliable references can help stop the spread of faulty information. Indeed, this highly-cited paper in Cell, which was retracted in 2015, continues to be cited by others uncritically as recently as this year.

Our new reference check may have prevented this. Running these citing papers through the scite reference check, something that could have been done at the time of writing or submission, readily identifies that the authors have cited a retracted article.

The reference check is not just about identifying retracted references as it can also help you contextualize your references. This paper, which has been cited over 200 times concludes “AICAR and 17-AAG, especially when combined, also show efficacy against aneuploid human cancer cell lines.” Of these citations, 11 indicate they support these findings with citation statements such as the following from 2018 and 2019.

However, there are also dissenting earlier findings from 2012 and 2014.

These debates and differences in findings are important to understand as well as address and using the scite reference check can now easily surface the context of references better.

The scite Reference Check saves time

Checking manuscripts and references like the above examples manually would be exceedingly difficult and time-consuming. Indeed, if an average manuscript has 35 references, and each reference is cited an average of 8 times, that means that there are 315 (35+(35x8)) relevant articles to check to see how they have been cited and if there are any editorial notices. If it takes 1.5 hours to read each article that is 472.5 hours or roughly 20 full days to read all directly related work. That is almost a month to check references against other references! This is so impractical that it is effectively impossible.

The scite Reference Check allows you to instantly see how your references have been cited by other articles. Simply log in, upload a PDF, and we’ll automatically parse the citations and references showing you how each reference was used in the paper, and if it has been supported or disputed by others or if it has been retracted, received an editorial expression of concern or some other notice. We’re making this functionality available to individuals as well as integrating it into major submission systems now.

Try it out, explore this example report, or watch this short demo: