The publication and citation impact profiles of Angewandte Chemie and the Journal of the American Chemical Society based on the sections of Chemical Abstracts: A case study on the limitations of the Journal Impact Factor

Abstract

The Journal Impact Factor (JIF) published by Thomson Reuters is often used to evaluate the significance and performance of scientific journals. Besides methodological problems with the JIF, the critical issue is whether a single measure is sufficient for characterizing the impact of journals, particularly the impact of multidisciplinary and wide‐scope journals that publish articles in a broad range of research fields. Taking Angewandte Chemie International Edition and the Journal of the American Chemical Society as examples, we examined the two journals' publication and impact profiles across the sections of Chemical Abstracts and compared the results with the JIF. The analysis was based primarily on Communications published in Angewandte Chemie International Edition and the Journal of the American Chemical Society during 2001 to 2005. The findings show that the information available in the Science Citation Index is a rather unreliable indication of the document type and is therefore inappropriate for comparative analysis. The findings further suggest that the composition of the journal in terms of contribution types, the length of the citation window, and the thematic focus of the journal in terms of the sections of Chemical Abstracts has a significant influence on the overall journal citation impact. Therefore, a single measure of journal citation impact such as the JIF is insufficient for characterizing the significance and performance of wide‐scope journals. For the comparison of journals, more sophisticated methods such as publication and impact profiles across subject headings of bibliographic databases (e.g., the sections of Chemical Abstracts) are valuable.

What makes a great journal great in the sciences? Which came first, the chicken or the egg?

“…They support the view of Seglen (1997) and Neuhaus et al (2009), among many others, that the impact factor (specifically, 2YIF, though similar comments would also apply to 5YIF, and possibly also 2YIF*), should not be used uncritically. We would temper such a recommendation that impact factors should not be used uncritically, as they do provide descriptive statistics that can be informative.…”

“…The authors conclude that impact and quality are not equivalent, that several bibliometric indicators should be used, that comparisons across disciplines should not be made, that bibliometric indicators should be complemented by other measures, especially peer assessment, for purposes of evaluating research output by individuals and research teams, and that the temptation to rely on automated evaluation based on bibliometric indicators should be resisted. Neuhaus et al (2009) also discuss the limitations of the journal impact factor, with an emphasis on chemistry. The critical issue investigated is whether a single measure is appropriate for characterizing journal impact in a multidisciplinary setting.…”

Empirical study of journal impact factors obtained using the classical two-year citation window versus a five-year citation window

“…One of the criticisms is that the 2-year citation window is too brief to capture all relevant scientific impact. The length of the JIF citation window seems too short for fields that do not reach maximum impact until 3 years or longer after publication (Neuhaus et al 2009). …”

“…By definition, these citations have no effect on the JIF although it is clear that the cited documents, despite their age, have some kind of impact on the authors of the citing document. Some authors have studied the effect on the JIF of changes in the citation window (for example, Glanzel and Moed 2002;Vanclay 2009;Neuhaus et al 2009). In addition, Amin and Mabe studied the average 2-and 5-year impact factors for around 200 chemistry journals.…”

The luck of the referee draw: the effect of exchanging reviews

“…The immediacy and the short length of the reports of research findings are hence important features of Communications, distinguishing them from research articles and reviews. 10 From 1984 to 2007 the number of Communications submitted to AC-IE rose from 449 to 5,489 5,7 (and this trend is continuing); thus when reviewing Communications the editors are under considerable pressure to reach rapid publication decisions.…”