Global investment in biomedical research has grown significantly over the last decades, reaching approximately a quarter of a trillion US dollars in 2010. However, not all of this investment is distributed evenly by gender. It follows, arguably, that scarce research resources may not be optimally invested (by either not supporting the best science or by failing to investigate topics that benefit women and men equitably). Women across the world tend to be significantly underrepresented in research both as researchers and research participants, receive less research funding, and appear less frequently than men as authors on research publications. There is also some evidence that women are relatively disadvantaged as the beneficiaries of research, in terms of its health, societal and economic impacts. Historical gender biases may have created a path dependency that means that the research system and the impacts of research are biased towards male researchers and male beneficiaries, making it inherently difficult (though not impossible) to eliminate gender bias. In this commentary, we – a group of scholars and practitioners from Africa, America, Asia and Europe – argue that gender-sensitive research impact assessment could become a force for good in moving science policy and practice towards gender equity. Research impact assessment is the multidisciplinary field of scientific inquiry that examines the research process to maximise scientific, societal and economic returns on investment in research. It encompasses many theoretical and methodological approaches that can be used to investigate gender bias and recommend actions for change to maximise research impact. We offer a set of recommendations to research funders, research institutions and research evaluators who conduct impact assessment on how to include and strengthen analysis of gender equity in research impact assessment and issue a global call for action.

As governments, funding agencies and research organisations worldwide seek to maximise both the financial and non-financial returns on investment in research, the way the research process is organised and funded is becoming increasingly under scrutiny. There are growing demands and aspirations to measure research impact (beyond academic publications), to understand how science works, and to optimise its societal and economic impact. In response, a multidisciplinary practice called research impact assessment is rapidly developing. Given that the practice is still in its formative stage, systematised recommendations or accepted standards for practitioners (such as funders and those responsible for managing research projects) across countries or disciplines to guide research impact assessment are not yet available.In this statement, we propose initial guidelines for a rigorous and effective process of research impact assessment applicable to all research disciplines and oriented towards practice. This statement systematises expert knowledge and practitioner experience from designing and delivering the International School on Research Impact Assessment (ISRIA). It brings together insights from over 450 experts and practitioners from 34 countries, who participated in the school during its 5-year run (from 2013 to 2017) and shares a set of core values from the school’s learning programme. These insights are distilled into ten-point guidelines, which relate to (1) context, (2) purpose, (3) stakeholders’ needs, (4) stakeholder engagement, (5) conceptual frameworks, (6) methods and data sources, (7) indicators and metrics, (8) ethics and conflicts of interest, (9) communication, and (10) community of practice.The guidelines can help practitioners improve and standardise the process of research impact assessment, but they are by no means exhaustive and require evaluation and continuous improvement. The prima facie effectiveness of the guidelines is based on the systematised expert and practitioner knowledge of the school’s faculty and participants derived from their practical experience and research evidence. The current knowledge base has gaps in terms of the geographical and scientific discipline as well as stakeholder coverage and representation. The guidelines can be further strengthened through evaluation and continuous improvement by the global research impact assessment community.

Although questionnaires are widely used in research impact assessment, their metric properties are not well known. Our aim is to test the internal consistency and content validity of an instrument designed to measure the perceived impacts of a wide range of research projects. To do so, we designed a questionnaire to be completed by principal investigators in a variety of disciplines (arts and humanities, social sciences, health sciences, and information and communication technologies). The impacts perceived and their associated characteristics were also assessed. This easy-to-use questionnaire demonstrated good internal consistency and acceptable content validity. However, its metric properties were more powerful in areas such as knowledge production, capacity building and informing policy and practice, in which the researchers had a degree of control and influence. In general, the research projects represented an stimulus for the production of knowledge and the development of research skills. Behavioural aspects such as engagement with potential users or mission-oriented projects (targeted to practical applications) were associated with higher social benefits. Considering the difficulties in assessing a wide array of research topics, and potential differences in the understanding of the concept of ‘research impact’, an analysis of the context can help to focus on research needs. Analyzing the metric properties of questionnaires can open up new possibilities for validating instruments used to measure research impact. Further to the methodological utility of the current exercise, we see a practical applicability to specific contexts where multiple discipline research impact is requires.

No abstract

The study concludes that, globally, the prioritization system for PKA has been implemented but had no effect on the prioritization of patients based on their severity. Nevertheless, in some centres, a moderate correlation between the order of operations performed and the priority score was identified.

This article presents the ex-post assessment of a program of clinical and health services research and the evaluation of the social impact. The Catalan Agency for Health Information, Assessment, and Quality (CAHIAQ) promotes a biannual open, public, competitive extramural research call to conduct non-commercial clinical and health services research. Its aim is to address local needs of research (knowledge gaps) and to assess the implementation of innovation. Approximately 5.8 million Euros have been allocated to the call. To meet the Agency's mission, a periodical 'call for expressions of interest' and topic prioritization is organized prior to the research call. The awarded projects are submitted to an ex-ante, ongoing, and ex-post assessment. Impact assessment of the research call on advancing knowledge and healthcare decision making is based on the Canadian Academy of Health Sciences framework (Panel on Return on Investment in Health Research, 2009). The methods used include bibliometric analysis, surveys to researchers and decision-makers, and a more in-depth case study of translation pathways. This includes a crossover of cases from 1996 to 2004. Some results are compared against other international health services research calls. The conclusion is that local agencies can significantly contribute to fill knowledge gaps in a specific context. Assessment of the complete research cycle provides opportunities for improving the entire research process (identification of knowledge needs, call for proposals, funding allocation, research completion, subsequent impact). Specifically, assessment of the different types of impact of research development on knowledge generation and decision making closes the evaluation cycle fulfilling the Agency's mission.

BackgroundThis article reports on the impact assessment experience of a funding program of non-commercial clinical and health services research. The aim was to assess the level of implementation of results from a subgroup of research projects (on respiratory diseases), and to detect barriers (or facilitators) in the translation of new knowledge to informed decision-making.MethodsA qualitative study was performed. The sample consisted of six projects on respiratory diseases funded by the Agency for Health Quality and Assessment of Catalonia between 1996 and 2004. Semi-structured interviews to key informants including researchers and healthcare decision-makers were carried out. Interviews were recorded, transcribed verbatim and analysed on an individual (key informant) and group (project) basis. In addition, the differences between achieved and expected impacts were described.ResultsTwenty-three semi-structured interviews were conducted. Most participants indicated changes in health services or clinical practice had resulted from research. The channels used to transfer new knowledge were mainly conventional ones, but also in less explicit ways, such as with the involvement of local scientific societies, or via debates and discussions with colleagues and local leaders. The barriers and facilitators identified were mostly organizational (in research management, and clinical and healthcare practice), although there were also some related to the nature of the research as well as personal factors. Both the expected and achieved impacts enabled the identification of the gaps between what is expected and what is truly achieved.ConclusionsIn this study and according to key informants, the impact of these research projects on decision-making can be direct (the application of a finding or innovation) or indirect, contributing to a more complex change in clinical practice and healthcare organization, both having other contextual factors. The channels used to transfer this new knowledge to clinical practice are complex. Local scientific societies and the relationships between researchers and decision-makers can play a very important role. Specifically, the relationships between managers and research teams and the mutual knowledge of their activity have shown to be effective in applying research funding to practice and decision-making. Finally the facilitating factors and barriers identified by the respondents are closely related to the idiosyncrasy of the human relations between the different stakeholders involved.

No abstract