Scientific evaluation of the scholarly publications

Abstract

Worthiness of any scientific journal is measured by the quality of the articles published in it. The Impact factor (IF) is one popular tool which analyses the quality of journal in terms of citations received by its published articles. It is usually assumed that journals with high IF carry meaningful, prominent, and quality research. Since IF does not assess a single contribution but the whole journal, the evaluation of research authors should not be influenced by the IF of the journal. The h index, g index, m quotient, c index are some other alternatives to judge the quality of an author. These address the shortcomings of IF viz. number of citations received by an author, active years of publication, length of academic career and citations received for recent articles. Quality being the most desirable aspect for evaluating an author's work over the active research phase, various indices has attempted to accommodate different possible variables. However, each index has its own merits and demerits. We review the available indices, find the fallacies and to correct these, hereby propose the Original Research Performance Index (ORPI) for evaluation of an author's original work which can also take care of the bias arising because of self-citations, gift authorship, inactive phase of research, and length of non-productive period in research.

Helping doctoral students crack the publication code: An evaluation and content analysis of the Australasian Journal of Educational Technology

“…The Google h-index and the 5-year impact factor have both grown in the past 2 years (Bond & Buntins, 2018); however citations alone are not enough to measure a journal's quality . Using impact factors to evaluate educational research is problematic due to bias, skewing of metrics, editorial policies, and methodological preferences (Lavie, 2009;Ouimet, Bedard, & Gelineau, 2011;Saxena, Thawani, Chakarbarty, & Gharpure, 2013;Staller, 2017), therefore this research seeks to explore AJET's impact and support of scholars and scholarship beyond metrics alone (West & Rich, 2012).…”

“…Whilst citation analyses are "commonly used to explore the intellectual structure of a given discipline" (Liu & Wang, 2005, p. 308), interpreting educational research quality using impact factors alone is problematic (Lavie, 2009;Ouimet, et al, 2011;Saxena, et al, 2013;Staller, 2017), particularly given the issues in assessing research quality in Australia (e.g., Smith, 2014). Therefore, in order to explore how AJET has furthered scholarship in the field (research question two), an analysis of AJET's rigour, impact and prestige was undertaken, following the framework by West & Rich (2012).…”

An analysis of the Australasian Journal of Educational Technology 2013-2017

“…Interpreting educational research quality by using impact factors has been identified as particularly problematic, due to a range of factors including skewing of metrics, editorial policy, the small number of journals indexed in the ISI (Lavie, 2009;Saxena, Thawani, Chakrabarty, & Gharpure, 2013;Staller, 2017;West & Rich, 2012), and author epistemological or methodological preferences (Ouimet, Bedard, & Gelineau, 2011). Therefore, the results of this analysis should be viewed as one measure of AJET's quality.…”

Knee Society Award Papers Are Highly Cited Works

“…Indeed, many additional factors may bias paper citation rates. These may include eminence of the author or institution [12,18]; tendency to self-citation by some authors [14]; increased citing of authors published in the same country or language [7,11]; and research with positive results rather than no difference or negative results [3,4]. Probably most important among these other factors is the prominence of the journal in which the paper is published.…”