Societal output and use of research performed by health research groups

Abstract

The last decade has seen the evaluation of health research pay more and more attention to societal use and benefits of research in addition to scientific quality, both in qualitative and quantitative ways. This paper elaborates primarily on a quantitative approach to assess societal output and use of research performed by health research groups (societal quality of research). For this reason, one of the Dutch university medical centres (i.e. the Leiden University Medical Center (LUMC)) was chosen as the subject of a pilot study, because of its mission to integrate top patient care with medical, biomedical and healthcare research and education. All research departments were used as units of evaluation within this university medical centre.The method consisted of a four-step process to reach a societal quality score per department, based on its (research) outreach to relevant societal stakeholders (the general public, healthcare professionals and the private sector). For each of these three types of stakeholders, indicators within four modes of communication were defined (knowledge production, knowledge exchange, knowledge use and earning capacity). These indicators were measured by a bottom-up approach in a qualitative way (i.e. all departments of the LUMC were asked to list all activities they would consider to be of societal relevance), after which they were converted into quantitative scores. These quantitative scores could then be compared to standardised scientific quality scores that are based on scientific publications and citations of peer-reviewed articles.Based on the LUMC pilot study, only a weak correlation was found between societal and scientific quality. This suggests that societal quality needs additional activities to be performed by health research groups and is not simply the consequence of high scientific quality. Therefore we conclude that scientific and societal evaluation should be considered to be synergistic in terms of learning for the future, accountability and advocacy.This quantitative approach to assess societal quality in a quantitative sense is based on indicators that function as proxies for society quality on different levels, based on the communication of researchers with their societal stakeholders (i.e. knowledge production, knowledge exchange and knowledge use). The methodology presented is just a first attempt to compare scientific quality scores (publication and citation scores) with societal quality scores in a quantitative way. This comparison can be used by organisations (e.g. university medical centres) in their planning and control cycle.

Achieving Research Impact Through Co‐creation in Community‐Based Health Services: Literature Review and Case Study

“…An emerging science of “societal impact assessment” seeks to categorize and measure both the various interactions between university academics and other stakeholders and the outputs and outcomes emerging from them . The European Seventh Framework Programme, for example, sought to assess synergies with science education, engagement with civil society and policymakers, dissemination to the general public in multiple languages, and the employment consequences of the research .…”

A review of the characteristics of 108 author-level bibliometric indicators

“…Dissemination in the public sphere counts publication and dissemination activities via other channels than academic books or articles. This indicator of publication count is just one of the indicators suggested by Mostert et al (2010) in their questionnaire tool to measure societal relevance which also includes standardised weighting schemes to accommodate certain activities in the field the researcher is active in. All the aforementioned counting methods assume an equal distribution of contribution across all authors of a publication.…”

From bench to bedside: The societal orientation of research leaders: The case of biomedical and health research in the Netherlands

“…14 This study found that entrepreneurial output is not significantly related to academic output. A study of societal quality in a UMC in the Netherlands (Mostert et al 2010) showed no relationship between societal and scientific quality. Finally, a French study (Jensen et al 2008) examined the relationship between popularisation activities (e.g.…”