Rankings in the Eyes of the Beholder: A Vox Populi Approach to Academic Journal Ranking

Abstract

The ranking of academic journals is a contentious issue in the current higher education environment. Across the world, peers judge academics for tenure and promotion on the basis of the quality or prestige of the journals in which they publish. This research proposes a new metric (i.e., the MAG score) to assess journal impact and ranking in the field of marketing using the vox populi approach. The findings show that the vox populi approach provides a more comprehensive measure of journal impact than other impact factor metrics from the perspective of academics.

An Update of the Vox Populi Approach to Academic Journal Rankings: 2011 in Review

“…Following data cleaning and verification, the final sample size was 1,005 cases (12.0%). This number of respondents was a significant (186%) increase from the previous study (Fam et al, 2011). The respondents were full-time academics from marketing, tourism or international business.…”

“…To further validate the MAG score, correlation analysis of the top 100 MAG score journals was undertaken against the SSCI and Scopus indices (Fam et al, 2011;Steward and Lewis, 2010), see Appendix C. In total 89 of the top 100 MAG Scholar journals are included in the SCOPUS index, while 66 are included in the SSCI database. As shown in Table 7, the correlation between the MAG scholar ranked journals and the Scopus, r(87) = .40, p < .01, and SSCI, r(64) = .34, p < .05, rankings are positive and significant.…”

“…One concern commonly voiced is that importance of emerging journals and attempting to rank them (Fam et al, 2011). A second concern is finding an acceptable method to capture the dynamic nature of journal rankings, as marketing priorities and activities move forward (Steward and Lewis, 2010).…”

“…The current paper only presents the journal ranking data for discussion. As in the previous study (Fam et al, 2011), the research attempts to understand the effect of journal ranking from the 'contributor' perspective, using the vox populi approach; that any judgment based on the intelligence of the masses will be free of passion and uninfluenced by rhetoric (Galton, 1907). More recently, research on the "wisdom of crowds" has found public opinion to be an effective predictive tool because of the capacity to synthesise large amounts of information and improve their judgement (Hastie and Kameda, 2005;Mannes, 2009;Soll and Larrick, 2009;Surowiecki, 2004).…”

Marketing, Tourism, and International Business Journal Rankings 2014: The Vox Populi Approach

“…The current paper only presents the journal ranking data for discussion. As in the previous two studies (Fam, Shukla, Sinha, Parackel and Chai 2011;Richard, Fam, Plimmer and Gerschewski 2012), the research endeavours to understand the effect of journal ranking from the "contributor" perspective, using the vox populi approach. This approach attempts to address the direction suggested by Steward and Lewis (2010) that "…efforts aimed at creating new perspectives on appraising the quality of journals in Marketing should be encouraged".…”