The Qualis system: a perspective from a multidisciplinary journal

Abstract

There is quite an amount of literature out there regarding the Qualis system and one wonders what someone can write about it that has not been discussed already. In this short essay, I will show the perspective of a multidisciplinary journal that has to live (or survive) in the very complex environment created by Qualis and the generated deleterious side effects for periodicals with broad scopes like that of the Annals of the Brazilian Academy of Sciences (AABC).The Qualis system was introduced in 1998 by CAPES (Coordenação de Aperfeiçoamento de Pessoal de Nível Superior), an agency of the Brazilian government whose purpose is to evaluate and foster the Brazilian graduate programs (e.g., Andrade and Galembeck 2009). Right from the beginning, it was clear that one of the major aspects of a graduate program are publications by both, researchers and students. Since the amount of produced papers is high, a system had to be developed to discriminate this production (e.g., Barata 2016). This is how Qualis got into the picture. Essentially, this system classifies publications in strata that are further used to evaluate a particular graduate program. This system has been changed along the years, and presently there are eight levels: A1 and A2, the highest; B1 to B5; and C, the lowest that does not add anything for the evaluation process. So far so good, if it were not for a small detail: the percentage of A1 + A2 cannot be more than 25% of all periodicals evaluated. Moreover, the percentage of A1 + A2 + B1 cannot be over half the periodicals that are taken into consideration in a given year (e.g., Rocha-e-Silva 2009). I will come back to this later.As known, the AABC is the sole periodical produced in Brazil that can be regarded as truly multidisciplinary, publishing articles from physics and mathematics, to paleontology and scientometrics, passing through the description of new species (biodiversity), research of new chemical compounds, studies that could lead to the cure of diseases, and new methodologies applied to engineering, just to cite a few. No wonder our journal appears in several different areas evaluated by CAPES, that are now slightly less than 50 (Barata 2016). Since 2010, the AABC is listed in 28 (2011) up to 39 (2016) areas with quite some variation regarding the strata. For example, in the last Qualis list (2016), the classification of the AABC fluctuates from A2 (e.g., Anthropology/ Archaeology; Agrarian Sciences) to B5 (Biological Sciences I and II). Just a reminder: B5 is close to nothing in the present evaluation system.

QUALIS: The journal ranking system undermining the impact of Brazilian science

“…The steeper decline in the number of citations per document since 2009, compared to the top Latin American countries publishing more scientific papers, suggest that the QUALIS journal ranking system has created incentives to publish in low-impact journals ranked highly by QUALIS. For instance, changes in the QUALIS journal rankings have affected submission rates in journals like Anais da Academia Brasileira de Ciências: Submissions from Biological Sciences plummeted after this subject area downgraded the journal from A2 to B2 in 2013 (Kellner, 2017). As faculty and graduate students are evaluated based on the number of papers they publish in journals that are highly ranked by QUALIS, they are more likely to select journals in the A categories with lower impact factors and higher acceptance rates (Aarssen et al, 2008).…”

“…Likewise, they reveal that a journal ranking system based on the realized impact of journals would avoid introducing distorted incentives, and thereby boost the global impact of Brazilian science (da Silva, 2009). It is also difficult to justify QUALIS as a mean to promote national journals in the age of open-access and pre-prints (Rocha-e-Silva, 2009b;Andriolo et al, 2010;Kellner, 2017), and less so if it is at the expense of the global impact of Brazilian science (Ferreira, Antoneli & Briones, 2013). QUALIS was once considered a temporary strategy (da Silva, 2009), and a recent report by CAPES has recommended it should not be used in the future any more, being replaced with "internationally established and broadly recognized metrics" (COMISSÃO ESPECIAL DE ACOMPANHAMENTO DO PNPG, 2020).…”

QUALIS: The journal ranking system undermining the impact of Brazilian science

“…Likewise, they reveal that a journal ranking system based on the realized impact of journals would avoid introducing distorted incentives, and thereby boost the global impact of Brazilian science (da Silva 2009). It is also difficult to justify QUALIS as a mean to promote national journals in the age of open-access and preprints (Andriolo et al 2010, Kellner 2017, Rocha-e-Silva 2009a and less so if it is at the expense of the global impact of Brazilian science (Ferreira et al 2013). QUALIS was once considered a temporary strategy (da Silva 2009), and a recent report by CAPES has recommended it should not be used in the future any more, being replaced with "internationally established and broadly recognized metrics" (Comissão Especial de Acompanhamento do PNPG 2020).…”

A brief summary of the impact and performance of different scientific fields at the AABC

“…From the 10 scientific areas considered by the AABC, Physical Sciences lacked any publication during 2018. The reduced number of submissions to this journal regarding this scientific field has already raised concerns in the past (Kellner 2017). The lowest IF restricted to an area was Social Sciences, followed by Engineering and Mathematical Sciences (Table I).…”