Identifying potential indicators to measure the outcome of translational cancer research: a mixed methods approach

Abstract

BackgroundIn a context where there is an increasing demand to evaluate the outcome of bio-medical research, our work aims to develop a set of indicators to measure the impact of translational cancer research. The objective of our study was to explore the scope and issues of translational research relevant to evaluation, explore the views of researchers on the evaluation of oncological translational research, and select indicators measuring the outcomes and outputs of translational research in oncology by consensus.MethodsSemi-structured interviews amongst 23 researchers involved in translational cancer research were conducted and analysed using thematic analysis. A two-round modified Delphi survey of 35 participants with similar characteristics was then performed followed by a physical meeting. Participants rated the feasibility and validity of 60 indicators. The physical meeting was held to discuss the methodology of the new indicators.ResultsThe main themes emerging from the interviews included a common definition for translational research but disagreements about the exact scope and limits of this research, the importance of multidisciplinarity and collaboration for the success of translational research, the disadvantages that translational research faces in current evaluation systems, the relative lack of pertinence of existing indicators, and propositions to measure translational cancer research in terms of clinical applications and patient outcomes. A total of 35 participants took part in the first round survey and 12 in the second round. The two-round survey helped us select a set of 18 indicators, including four that seemed to be particularly adapted to measure translational cancer research impact on health service research (number of biomarkers identified, generation of clinical guidelines, citation of research in clinical guidelines, and citation of research in public health guidelines). The feedback from participants helped refine the methodology and definition of indicators not commonly used.ConclusionIndicators need to be accepted by stakeholders under evaluation. This study helped the selection and refinement of indicators considered as the most relevant by researchers in translational cancer research. The feasibility and validity of those indicators will be tested in a scientometric study.Electronic supplementary materialThe online version of this article (doi:10.1186/s12961-015-0060-5) contains supplementary material, which is available to authorized users.

Application of Mixed Methods in Health Services Management Research: A Systematic Review

“…Studies used exploratory sequential designs mainly for instrument development, program evaluation, and description. As examples, Shore et al (2015) used semistructured telephone interviews and then quantitative clinical data from 67 Veterans Health Administration sites to identify site-level practices that were associated with high oral anticoagulant adherence; Thonon et al (2015) sequentially employed semistructured interviews and a modified Delphi survey with European cancer researchers to identify and select outcome indicators for translational oncology research. Both intervention and multistage designs also were employed in program evaluation studies.…”

Identifying organisational capability of hospitals amid the new healthcare reform in China: a Delphi study

“…The Delphi method is a structured process and a systematic, effective, reliable and comprehensive technique for collecting and distilling knowledge from a group of experts by means of a series of anonymous questionnaires interspersed with controlled feedback. [33][34][35] Therefore, this method is widely used for constructing index systems.…”

A RAND-Modified Delphi on Key Indicators to Measure the Efficiency of Living Kidney Donor Candidate Evaluations

“…We initiated word-of-mouth recruitment by leveraging existing networks known to the research team and relying on snowball sampling, a common technique used in qualitative research ( Figure 1) (17,18). We deliberately invited participants who represented different perspectives (patient, family, different types of health care professionals) across Canada (18).…”