The Five Stars of Online Journal Articles – an article evaluation framework

Abstract

I propose five factors-peer review, open access, enriched content, available datasets and machinereadable metadata-as the Five Stars of Online Journal Articles, a constellation of five independent criteria within a multi-dimensional publishing universe against which online journal articles can be evaluated, to see how well they match up to current visions for research communications. Achievement along each of these publishing axes can vary, analogous to the different stars within the constellation shining with varying luminosities. I suggest a five-point scale for each by which a journal article can be evaluated, and a diagrammatic representation for such evaluations. While the criteria adopted for these scales are somewhat arbitrary, and while the rating of a particular article on each axis may involve elements of subjective judgment, these Five Stars of Online Journal Articles provide a conceptual framework by which to judge the degree to which any article achieves or falls short of the ideal, which should be useful to authors, editors and publishers. I exemplify such evaluations using my own recent publications of relevance to semantic publishing.

A multi-disciplinary perspective on emergent and future innovations in peer review

“…The most simple definitions by McCormack (2009) and Mulligan et al (2008) presented OPR as a process that does not attempt “to mask the identity of authors or reviewers” ( McCormack, 2009 , p.63), thereby explicitly referring to open in terms of personal identification or anonymity. Ware (2011 , p.25) expanded on reviewer disclosure practices: “Open peer review can mean the opposite of double blind, in which authors’ and reviewers’ identities are both known to each other (and sometimes publicly disclosed), but discussion is complicated by the fact that it is also used to describe other approaches such as where the reviewers remain anonymous but their reports are published.” Other authors define OPR distinctly, for example by including the publication of all dialogue during the process ( Shotton, 2012 ), or running it as a publicly participative commentary ( Greaves et al , 2006 ).…”

Metrics for openness

“…Scholarly Publishing and Academic Resources Coalition (SPARC) () provide a six‐faceted approach to the evaluation of a journal's policies: reader rights, reuse rights, copyrights, author posting rights, automatic posting, and machine readability. Shotton () provides a five‐point scale which includes a distinction between access to read and wider rights of reuse.…”

“…We now present several metrics that illustrate alternative methods for characterizing the openness of the scholarly literature. We follow Shotton () in using our own work to demonstrate the ideas.…”

“…This measure is based on publishers’ agreements and so reflects potential accessibility rather than actual availability to information seekers. Shotton () includes open access in a five‐criteria framework for article‐level evaluation; distinguishing between articles with different reuse rights and different financing models.…”

Accelerating Scientists’ Knowledge Turns

“…the US NIH publicaccess.nih.gov). However, Open Access does not guarantee reusability [52], and though it is free for consumers it is not for providers, restricting publication to those who can afford it. As the majority of publishers remain subscription-based and legislative efforts in the USA aim to block open access mandates, open access for all content is some way off.…”