Accounting for Multiplicity in Inference on Economics Journal Rankings

Horrace, William C.; Parmeter, Christopher F.

doi:10.1002/soej.12219

Cited by 7 publications

(4 citation statements)

References 36 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The robustness of rankings of economics journals was studied by Seiler and Wohlrabe (2014), where the rankings were based on the JIF, by considering the skewness of the distribution of citations to the articles in a journal. Horrace and Parmeter (2017) revisited the Stern study (2013) referred to above, but performed multivariate inference to control the overall error rate of the testing procedure. Confidence intervals around values of the PP top 10% indicator, i.e.…”

Section: Introductionmentioning

confidence: 99%

Uncertainty and the ranking of economics journals

Lyhagen

Ahlgren

2020

Scientometrics

View full text Add to dashboard Cite

Journal rankings often show significant changes compared to previous rankings. This gives rise to the question of how well estimated the rank of a journal is. In this contribution, we consider uncertainty in a ranking of economics journals. We use the invariant method of Pinski and Narin to rank the journals. We propose an uncertainty measure, which is based on a bootstrap approach. The measure is the average absolute change in rank, which we see as a reasonable uncertainty measure regarding rankings. We further calculate, based on the bootstrap method, 95% confidence interval for the observed values of the invariant method. We show that ranks of the highest, as well as the lowest, ranked journals are well estimated, while there is a high degree of uncertainty regarding the rank of many mid-ranked journals. The distribution of the underlying measure is useful for identifying groups of journals that are more or less of the same quality (from the point of view of the invariant measure). The journal with the highest observed value of the invariant measure, Journal of Political Economy, has the best performance and constitutes a singleton, whereas Quarterly Journal of Economics and Econometrica form the next group (there is a slight overlap between the two with respect to confidence intervals). The journals ranked between about 190–230 form another group in which there are no major quality differences between the journals, as the confidence intervals are overlapping.

show abstract

Section: Introductionmentioning

confidence: 99%

Uncertainty and the ranking of economics journals

Lyhagen

Ahlgren

2020

Scientometrics

View full text Add to dashboard Cite

show abstract

“…We do not report the contextual effect estimates in our results below to save space (these estimates are available upon request from the authors). 29 See Horrace and Parmeter (2017), who recently apply ranking and selection to economics journal citation counts to determine a subset of the 'best' journals. least 1 ↵.…”

Section: Empirical Modelmentioning

confidence: 99%

What Makes a Classmate a Peer? Examining Which Peers Matter in NYC Elementary Schools

Horrace

Jung²,

Presler³

et al. 2021

SSRN Journal

View full text Add to dashboard Cite

“…In the literature, several studies propose new or updated ranking methods either at specific discipline- or field levels, or across all academic journals (Bornmann et al , 2016; Carraher and Paridon, 2015; Gursoy and Sandstrom, 2016; Horrace and Parmeter, 2017; Mahmood, 2017; Meese et al , 2017; Okumus et al , 2017; Tourish and Willmott, 2015; Tüselmann et al , 2015). These studies have primarily relied on either the basic or advanced formulations, including the citation count of journals or the stated preferences or expertise obtained via surveys.…”

Section: Introductionmentioning

confidence: 99%

A new approach to journal ranking: social structure in hospitality and tourism journals

Köseoğlu

2018

IJCHM

View full text Add to dashboard Cite

Purpose This study introduces a new approach, called the social structure approach, for ranking academic journals by focusing on hospitality and tourism journals; and a hybrid metric, including the combination of the journal impact factor via citations and a social network metric, called the journal knowledge domain index (JKDI). Design/methodology/approach Twenty-five hospitality and tourism journals were selected to test this approach. Collaboration-based metrics, productivity-based metrics, and network-based metrics are considered components of the social structure approach. Additionally, a hybrid metric, including the combination of the journal impact factor via citations and a social network metric, JKDI, is developed. Findings The study’s findings show that top or leading journals have a weaker position in some social structure approach metrics compared to other (or follower) journals. However, according to the JKDI, leading journals have remained constant with the other ranking studies. Practical implications The ranking of academic journals is vital for the stakeholders of academia. Consequently, the findings of this study may help stakeholders to design an optimal ranking system and formulate and implement effective research strategies for knowledge creation and dissemination. Originality/value As one of the first in the journal-ranking literature, this study has significant implications, as it introduces a new ranking approach.

show abstract

Accounting for Multiplicity in Inference on Economics Journal Rankings

Cited by 7 publications

References 36 publications

Uncertainty and the ranking of economics journals

Uncertainty and the ranking of economics journals

What Makes a Classmate a Peer? Examining Which Peers Matter in NYC Elementary Schools

A new approach to journal ranking: social structure in hospitality and tourism journals

Contact Info

Product

Resources

About