No abstract

Scholarly peer review is a complex collaborative activity that is increasingly supported by web-based systems. Yet little is known about how reviewers and authors interact in such environments, how criticisms are conveyed, or how the systems may affect the interactions and use of language of reviewers and authors. We looked at one aspect of the interactions between reviewers and authors, the use of politeness in reviewers' comments. Drawing on Brown and Levinson's politeness theory, we analyzed how politeness strategies were employed by reviewers to mitigate their criticisms in an open peer review process of a special track of a Human-Computer Interaction conference. We found evidence of frequent use of politeness strategies and that open peer review processes hold unique challenges and opportunities for using politeness strategies. Our findings revealed that less experienced researchers tended to express unmitigated criticism more often than did experienced researchers, and that reviewers tended to use more positive politeness strategies (e.g., compliments) toward less experienced authors. Based on our findings, we discuss implications for research communities, and the design of peer reviewing processes and the information systems that support them.

Design of peer‐review support systems is shaped by the policies that define and govern the process of peer review. An important component of these are policies that deal with anonymity: The rules that govern the concealment and transparency of information related to identities of the various stakeholders (authors, reviewers, editors, and others) involved in the peer‐review process. Anonymity policies have been a subject of debate for several decades within scholarly communities. Because of widespread criticism of traditional peer‐review processes, a variety of new peer‐review processes have emerged that manage the trade‐offs between disclosure and concealment of identities in different ways. Based on an analysis of policies and guidelines for authors and reviewers provided by publication venues, we developed a framework for understanding how disclosure and concealment of identities is managed. We discuss the appropriate role of information technology and computer support for the peer‐review process within that framework.

Facilitating the reuse process and enabling unanticipated reuse can improve efficiency of distributed collaboration. However, supporting the reuse process in complex and dynamic contexts, where future use of information is difficult to predict, is challenging. Collaborative analytics exemplifies such a context. We employed distributed cognition theory to design a collaborative visual analytics system, called AnalyticStream, for facilitating the reuse of analysis outcomes. In contrast with the commonly used detail-oriented approach to applying distributed cognition, we performed a high level analysis of the design situation and we identified the cognitive processes that could be distributed over people to facilitate their collaboration. We examined some of the ideas derived from the theoretical analysis, by designing a simple reminding process through recommending relevant pieces of analysis, as well as a mechanism for attention management through allowing users greater control over their shared activity streams. A mixed-methods study of AnalyticStream showed that suggesting relevant artefacts facilitated discovering and consequently reusing them, and provided context-relevant awareness of other analysts' activities.

Recruiting qualified reviewers, though challenging, is crucial for ensuring a fair and robust scholarly peer review process. We conducted a survey of 307 reviewers of submissions to the International Conference on Human Factors in Computing Systems (CHI 2011) to gain a better understanding of their motivations for reviewing. We found that encouraging high-quality research, giving back to the research community, and finding out about new research were the top general motivations for reviewing. We further found that relevance of the submission to a reviewer's research and relevance to the reviewer's expertise were the strongest motivations for accepting a request to review, closely followed by a number of social factors. Gender and reviewing experience significantly affected some reviewing motivations, such as the desire for learning and preparing for higher reviewing roles. We discuss implications of our findings for the design of future peer review processes and systems to support them.