Evaluating the quality of research publications: A pilot study of school librarianship

Abstract

Evaluation of research quality is becoming more important in the field of library and information science, as in other fields. This pilot study is a preliminary attempt to address issues associated with determining the quality of the published research in one area of library and information science, specifically school librarianship. The main aims were (1) to test the extent to which experienced evaluators agreed in their rankings of published research articles on the basis of quality and (2) to investigate the approaches to evaluation used by the experienced evaluators. A qualitative, naturalistic research design was used. On the basis of a comprehensive literature review, four approaches to evaluation were identified; they were generally supported through an analysis of the responses of the experienced evaluators. However, although the majority of the evaluators agreed on the article ranked lowest, basic statistical analyses showed less agreement about the other articles. Although subject knowledge (of the field of school librarianship in this case) may have some influence on the evaluations, cluster analysis suggests that there may be differences in the value perceptions of the evaluators that also carry weight. More research would be needed to gain a better understanding of these value perceptions and their relationship, if any, to the four approaches to evaluation that were identified through the literature.

The Open Source School Library Research Database: Learning without Borders

“…To honour Dr. Clyde, a team at the University of Alberta has been working to recreate her database and make it available to all. This presentation continues the work of Asselin (2011), Clyde (2006Clyde ( , 2004Clyde ( , 2003Clyde ( , 2002Clyde ( , 2001Clyde ( , 1996aClyde ( , 1996b and Mardis (2011).…”

“…(p. 1120) Manzari (2013) examined LIS journal prestige by surveying full-time faculty in American Library Association (ALA)-accredited programs. Clyde's (2006Clyde's ( , 2004Clyde's ( , 2003Clyde's ( , 2002Clyde's ( , 2001Clyde's ( , 1996aClyde's ( , 1996b previous work clearly demonstrates that most of the research in school librarianship has been published in two major publications (1) School Libraries Worldwide and (2) School Library Research (formerly School Library Media Research). Clyde and Oberg (2004) in their study of research in School Libraries Worldwide between 1995-2003 found that about 30% of research articles were from the USA.…”

“…Researchers interested in quality research journal articles tend to look to the quality of the journal itself. Clyde (2004) lists the following strategies for assessing the quality of the journal:…”

Evidence or Evidence Based Practice? An Analysis of IASL Research Forum Papers, 1998-2009

“…Because RCTs involve the use of randomly assigned participants in control groups and treatment groups, to deprive children of a possibly beneficial educational intervention by including them in a control group raises ethical concerns as well (Morrison, 2001). Therefore, researchers have sought other methods to assess the soundness of school library research (e.g., Clyde, 2004) and its ability to inform practice that benefits children's learning (Todd, 2009) to remain relevant in the educational policy context (Booth, 2003). To date, these alternate methods of establishing the soundness of research have been based on an investigation of the peer review process (Clyde 2004(Clyde , 2006 and the evaluation of the outcomes of practice based on research (Todd, 2009).…”

“…Therefore, researchers have sought other methods to assess the soundness of school library research (e.g., Clyde, 2004) and its ability to inform practice that benefits children's learning (Todd, 2009) to remain relevant in the educational policy context (Booth, 2003). To date, these alternate methods of establishing the soundness of research have been based on an investigation of the peer review process (Clyde 2004(Clyde , 2006 and the evaluation of the outcomes of practice based on research (Todd, 2009). As the use of EBP has become more common in LIS, it has evolved into a type of cycle in which the outcomes of the evidence based decision have become the object of study and, in essence, project and performance evaluations have become data sources.…”