Aggregating post-publication peer reviews and ratings

Abstract

Allocating funding for research often entails the review of the publications authored by a scientist or a group of scientists. For practical reasons, in many cases this review cannot be performed by a sufficient number of specialists in the core domain of the reviewed publications. In the meanwhile, each scientist reads thoroughly, on average, about 88 scientific articles per year, and the evaluative information that scientists can provide about these articles is currently lost. I suggest that aggregating in an online database reviews or ratings on the publications that scientists read anyhow can provide important information that can revolutionize the evaluation processes that support funding decisions. I also suggest that such aggregation of reviews can be encouraged by a system that would provide a publicly available review portfolio for each scientist, without prejudicing the anonymity of reviews. I provide some quantitative estimates on the number and distribution of reviews and ratings that can be obtained.

Deep impact: unintended consequences of journal rank

“…One avenue to move in this direction may be the recently announced Episcience Project (Van Noorden, 2013). Other solutions certainly exist (Bachmann, 2011; Birukou et al, 2011; Kravitz and Baker, 2011; Kreiman and Maunsell, 2011; Zimmermann et al, 2011; Beverungen et al, 2012; Florian, 2012; Ghosh et al, 2012; Hartshorne and Schachner, 2012; Hunter, 2012; Ietto-Gillies, 2012; Kriegeskorte, 2012; Kriegeskorte et al, 2012; Lee, 2012; Nosek and Bar-Anan, 2012; Pöschl, 2012; Priem and Hemminger, 2012; Sandewall, 2012; Walther and Van den Bosch, 2012; Wicherts et al, 2012; Yarkoni, 2012), but the need for an alternative system is clearly pressing (Casadevall and Fang, 2012). Given the data we surveyed above, almost anything appears superior to the status quo.…”

Problems with Traditional Science Publishing and Finding a Wider Niche for Post-Publication Peer Review

“…The use of copyrighted protocols or data may also pose some interesting legal challenges (Stodden, 2009) but how are these to be addressed when dealing with a global peer pool? Florian (2012) provides an interesting, unique, and in fact rather simple but potentially effective way to agglomerate peer potential in PPPR. The author estimates that on average, a scientist reads 88 papers a year (although this can be as much as 480 or more for tenured professors holding editorial positions) but that, at the end of each year, the opinions and information gathered about each of those papers becomes lost and is never used in any productive way.…”

Is There Any Interest Among Iranian Biomedical Journals and Researchers to Publish a “Letter to the Editors”?

“…In fact, if the editorial board accepts the "letter to the editor" which points out the methodological errors of a previously published article, it implicitly denotes that publishing such an article may damage the journal reputation (7,8).…”