Peer review and theCurrent Anthropologyexperience

The Reliability of Peer Reviews of Papers on Information Systems

“…This leads on to a number of suggestions: training reviewers to be more thorough, deliberately choosing reviewers likely to disagree so that the comments authors receive are more varied, and increasing the number of reviewers so that more points are likely to be picked up. According to one of the commentators on the article by Peters and Ceci [13], the journal Current Anthropology solicited up to 15 reviews of each article for this purpose, and because "dependence on two or three referees is ... downright dangerous". If reviewers' comments are regarded as providing a sample of the population of possible points to be made about the paper, the problem is that of deciding how large the sample of reviewers needs to be to achieve a "reasonable coverage" of this population -the framework and methods developed by Wood and Christy [14,15] could be used for this purpose.…”

Quality in research and the significance of community assessment and peer review: education’s idiosyncrasy

“…Belshaw (1982), for example, suggested that larger numbers of reviewers be used to improve on poor reliability engendered by reviewer bias or incompetence. Blackburn and Hakel (2006) suggest providing reviewers with better anchors for rating scales which should increase the reliability of ratings by providing them with a common set of criteria for their ratings.…”

Making Accounting Knowledge: Peering at Power

“…( Peer review is practiced by nearly every academic discipline Belshaw, ) 1982 . The scrutiny of informed reviewers is the test that separates those that try to publish from those that are successful publishers.…”