Carta. Por un cambio en la evaluación de la investigación científica: “no tirar al niño junto con el agua sucia”

Codina, Lluís

doi:10.3145/epi.2021.jul.18

Cited by 4 publications

(2 citation statements)

References 4 publications

(3 reference statements)

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…In addition, the active debate on how to appropriately evaluate researchers and research groups has been promoted by the academic world [5][6][7][8][9][10][11], and evaluation agencies themselves (ANECA, in Spain) display an active interest in reflecting on said aspects.…”

Section: Introductionmentioning

confidence: 99%

Habits and perceptions regarding open science by researchers from Spanish institutions

et al. 2023

View full text Add to dashboard Cite

The article describes the results of the online survey on open science (OS) carried out on researchers affiliated with universities and Spanish research centres and focused on open access to scientific publications, the publication process, the management of research data and the review of open articles. The main objective was to identify the perception and habits of researchers with regard to practices closely linked to open science and the scientific value added is that offers an in-depth picture of researchers as one of the main actors to whom this transformation and implementation of open science will fall. It focuses on the different aspects of OS: open access, open data, publication process and open review in order to identify habits and perceptions. This is to make possible an implementation of the OS movement. The survey was carried out among researchers who had published in the years 2020–2021, according to data obtained from WoS. It was emailed to a total of 8,188 researchers and obtained a total of 666 responses, of which 554 were complete, the rest being forms with some questions unanswered. The main results showed that open access still requires the diffusion of practices and services provided by the institution, as well as training (library or equivalent service) and institutional support from the competent authorities (vice rectors or equivalent) in specific aspects such as data management. In the case of data, around 50% of respondents stated they had stored data in a repository, and of all the options, the most frequently given was that of an institutional repository, followed by a discipline repository. Among the main reasons for doing this, we found transparency, visibility of data and the ability to validate results. For those who stated they had never stored data, the most frequent reasons for not having done so were privacy and confidentiality, the lack of a mandated data policy or a lack of knowledge of how to do it. In terms of open peer review, participants mentioned a certain reticence to the opening of evaluations due to potential conflicts of interest that may arise or because lower-quality content might be accepted in order to avoid conflicts. In addition, the hierarchical structure of senior researcher versus junior researcher might affect reviews. The main conclusions indicate a need for persuasion of OA to take place; APCs are an economic barrier rather than the main criterion for journal selection; OPR practices may seem innovative and emerging; scientific and evaluation policies seem to have a clear effect on the behaviour of researchers; researchers state that they share research data more for reasons of persuasion than out of obligation. Researchers do question the pathways or difficulties that may arise on a day-to-day basis and seem aware that we are undergoing change, where academic evaluation or policies related to open science, its implementation and habits among researchers may change. In this sense, more and better support is needed on the part of institutions and faculty support services.

show abstract

Section: Introductionmentioning

confidence: 99%

Habits and perceptions regarding open science by researchers from Spanish institutions

et al. 2023

View full text Add to dashboard Cite

show abstract

“…Palabras clave: Evaluación científica; Evaluación del rendimiento de los investigadores; Política científica; Indicadores bibliométricos; Profesorado universitario; Enseñanza universitaria; Sexenios de investigación; Aneca; España. Estos dos últimos años en España hemos asistido a intensos debates sobre el rumbo que debe seguir el sistema de evaluación académica (Delgado-López-Cózar; Ràfols; Abadal, 2021;2022;Molas-Gallart, 2022), tanto en lo que respecta al modelo de acreditación del profesorado como a la forma de evaluar la dimensión investigadora con especial énfasis en el uso de indicadores bibliométricos (Aguillo, 2021;Codina, 2021;Rodríguez-Navarro, 2022) Esto se produce en el contexto de la introducción de cambios en el sistema de educación superior en España (Arenilla-Sáez, 2021), con la aprobación de una nueva ley orgánica (Ahedo-Gurrutxaga; Martínez-Palacios; Ormazabal-Gaston, 2022) y de la germinación de un movimiento impulsado por la Comisión Europea para cambiar los sistemas de evaluación de la investigación en Europa (European Commission, 2021;2022).…”

unclassified

López-Cózar

2022

ThinKEPI

View full text Add to dashboard Cite

This essay provides some reflections on the origin of the current scientific evaluation system in Spain, focusing on an analysis of the system of selection and accreditation of university faculty. This system was put in place to prevent academic inbreeding. The thesis that this is mainly a centralized system is rebutted: although accreditation is indeed centralized, faculty selection is carried out by universities themselves. It is argued that we cannot fully trust universities to establish policies that are immune to localism and nepotism if they are not held fully responsible for their actions. The functional imbalance that exists in Spanish university teaching staff is highlighted. Among the activities currently carried out by university professors (teaching, research, knowledge transfer, and management), research is the most highly valued in evaluation programs. More specifically, publications are key to launch and advance careers in the academic environment. The success of the program of sexenios (six-year assessment periods), which serves not only as an incentive for productivity but also as an instrument to establish scientific reputation and academic hierarchy, has contributed to this situation. Other features of the current evaluation system in Spain are the predominance of individual-level evaluation and the use of bibliometric indicators as the final quality values. The Spanish academic evaluation system chose to give preference to individual-level evaluation. Once again, the origin of this approach can be found in the introduction of teacher and researcher productivity salary supplements in 1989. Spanish academics grew up in a culture of individual-level evaluation where they had to obtain their jobs through competitive examinations and were rewarded with bonuses on the basis of personal achievements. Quantitative criteria appeared as the necessary antidotes to curb nepotism. In a country lacking infrastructure and a culture of evaluation, quantitative indicators ensured objectivity and efficiency in the processes of selection and evaluation of publications, and, indirectly, a minimum amount of rigor and relevance of the research itself. The thoughts exposed here are accompanied by suggestions that might help guide the necessary reform of the scientific evaluation system. Resumen Reflexiones sobre el origen del sistema de evaluación científica en España, centrando el análisis en el sistema de selección y acreditación del profesorado universitario. Se recuerda cómo surgió para evitar la endogamia universitaria. Se desmonta la tesis de que estemos ante un sistema centralizado: la acreditación sí lo está, pero la selección del profesorado depende de las universidades. Se defiende que no podemos confiar plenamente en las universidades para que instauren políticas inmunes al localismo y el nepotismo si no se les hace plenamente responsables de sus actos. Se destaca el desequilibrio funcional que existe en el profesorado universitario español. Dentro de las actividades que actualmente se atribuyen al profesor universitario (docencia, investigación, transferencia, gestión) son las de investigación las más valoradas, especialmente las publicaciones que son claves para ingresar y progresar en la carrera académica. A ello ha contribuido el éxito de los sexenios de investigación no sólo como incentivo de productividad sino de reputación científica y jerarquización académica. Otros rasgos de nuestro sistema de evaluación son el predominio de la evaluación individual y el uso de indicadores bibliométricos como supremos valores de calidad. El sistema español de evaluación académica optó por dar preferencia a la evaluación individual. De nuevo el origen de esta inclinación hay que buscarla en la introducción de los complementos de productividad docente e investigador en 1989. Los académicos españoles crecieron en una cultura de evaluación individual donde debían obtener sus plazas por concursos-oposición y ser premiados con gratificaciones basadas en la consecución de méritos personales. Los criterios cuantitativos aparecieron como los antídotos necesarios para frenar el nepotismo. En un país sin infraestructura y cultura evaluativa aseguraban la objetividad y eficiencia en los procesos de selección y valoración de las publicaciones e indirectamente un mínimo rigor y relevancia de las propias investigaciones. Los pensamientos expuestos se acompañan de sugerencias que ayuden a orientar la necesaria reforma del sistema de evaluación científica.

show abstract

Letter: The unbearable lightness of the evaluation of research projects: do we need to regularize the professional activity of research evaluation?

Orduña-Malea

2021

EPI

View full text Add to dashboard Cite

This letter aims to highlight severe inconsistencies in the scientific evaluation of regionally publicly funded research projects, uncovering structural failures related to the public management of science that go beyond technical aspects of bibliometrics. This problem is exemplified through an illustrative case study (the public grants program to emerging research groups funded by the regional government of the Valencian region, Spain, NUTS-2), describing three complementary aspects of this call (the evaluation system, the evaluation process, and the communication of the evaluation). The problems illustrated by this case study lead to a general problem related to the nonstrategic use of science. Finally, the national and international regularization of the professional activity of research evaluation in general, and bibliometrics in particular, is proposed as a route to avoid mismanagement of science (and scientific evaluation). Resumen Esta carta tiene como objetivo resaltar graves inconsistencias en la evaluación científica de proyectos de investigación financiados con fondos públicos regionales, destapando fallos estructurales relacionados con la gestión pública de la ciencia que van más allá de los aspectos técnicos bibliométricos. Este problema se ejemplifica a través de un estudio de caso ilustrativo (el programa de ayudas públicas a grupos de investigación emergentes financiado por el gobierno regional de la Comunidad Valenciana, España, NUTS-2), describiendo tres aspectos complementarios de esta convocatoria (el sistema de evaluación, el proceso de evaluación, y la comunicación de la evaluación). Los problemas ilustrados por este estudio de caso conducen a un problema general relacionado con el uso no estratégico de la ciencia. Se propone finalmente la regularización nacional e internacional de la actividad profesional de la evaluación de la investigación en general, y de la bibliometría en particular, como vía para evitar la mala gestión de la ciencia (y de la evaluación científica). Palabras clave Evaluación de la investigación; Bibliometría; Proyectos de investigación; Financiación pública de la ciencia; Ciencia de la ciencia; Meta-investigación; Convocatorias de investigación regionales; Gestión de la ciencia.

show abstract

Carta. Por un cambio en la evaluación de la investigación científica: “no tirar al niño junto con el agua sucia”

Cited by 4 publications

References 4 publications

Habits and perceptions regarding open science by researchers from Spanish institutions

Habits and perceptions regarding open science by researchers from Spanish institutions

Ante la reforma del sistema espaÃ±ol de evaluaciÃ³n acadÃ©mica: recordando el pasado para no errar en el presente

Letter: The unbearable lightness of the evaluation of research projects: do we need to regularize the professional activity of research evaluation?

Contact Info

Product

Resources

About