Assessing excellence in translational cancer research: a consensus based framework

Abstract

BackgroundIt takes several years on average to translate basic research findings into clinical research and eventually deliver patient benefits. An expert-based excellence assessment can help improve this process by: identifying high performing Comprehensive Cancer Centres; best practices in translational cancer research; improving the quality and efficiency of the translational cancer research process. This can help build networks of excellent Centres by aiding focused partnerships. In this paper we report on a consensus building exercise that was undertaken to construct an excellence assessment framework for translational cancer research in Europe.MethodsWe used mixed methods to reach consensus: a systematic review of existing translational research models critically appraised for suitability in performance assessment of Cancer Centres; a survey among European stakeholders (researchers, clinicians, patient representatives and managers) to score a list of potential excellence criteria, a focus group with selected representatives of survey participants to review and rescore the excellence criteria; an expert group meeting to refine the list; an open validation round with stakeholders and a critical review of the emerging framework by an independent body: a committee formed by the European Academy of Cancer Sciences.ResultsThe resulting excellence assessment framework has 18 criteria categorized in 6 themes. Each criterion has a number of questions/sub-criteria. Stakeholders favoured using qualitative excellence criteria to evaluate the translational research “process” rather than quantitative criteria or judging only the outputs. Examples of criteria include checking if the Centre has mechanisms that can be rated as excellent for: involvement of basic researchers and clinicians in translational research (quality of supervision and incentives provided to clinicians to do a PhD in translational research) and well designed clinical trials based on ground-breaking concepts (innovative patient stratification, substantial fraction of phase I/II trials, investigator-initiated trials). Critically, the framework supports reduced bureaucracy by building on existing European evaluation systems.ConclusionsThe excellence framework is the product of an intense stakeholder consensus building exercise. It will be piloted during an expert peer review/site visit of at least three European Comprehensive Cancer Centres. The findings regarding content, governance and implementation can have relevance for other clinical and research fields.

Identifying potential indicators to measure the outcome of translational cancer research: a mixed methods approach

“…Compared with the study from Rajan et al [ 7 ], our study holds significant differences. Firstly, the Excellence Designation System proposed relies on peer review by experts, which requires organisation and participant input, while the set of indicators that we have developed can be measured using bibliometric data with minimal effort from participants.…”

“…Our work is part of a broad initiative to assess the quality of translational oncological research. Other researchers have focused on developing an assessment framework, called the Excellence Designation System, containing 18 criteria to identify excellent comprehensive cancer centres [ 7 ]. We aim to develop metrics to measure the outcome and impact of translational cancer research.…”

Markers of achievement for assessing and monitoring gender equity in translational research organisations: a rationale and study protocol

“…The fundamental rationale for this study stems from the desire of the participating NIHR BRCs to make their own measurable contribution to accelerating women's advancement and leadership in translational research. We extend previous work on performance assessment in translational research, 17–19 70–72 by focusing on gender equity. Our intention is to develop a new multidimensional conceptual framework for gender equity performance assessment in order to use it both for retrospective evaluation and prospective planning and monitoring with a view to accelerating women's advancement and leadership.…”

Quality assessments for cancer centers in the European Union

“…Another reason is that assessing impact of research on healthcare outcomes is more difficult than assessing care outcomes [ 34 ]. In research, metric-driven indicators such as impact factors are often criticized [ 35 ] and consensus on value-based indicators is still evolving e.g., how to define success in translational research (bench to bedside and back) in terms of practice-changing innovations [ 36 ]. The awareness that alignment between research and clinical areas is essential in successful translational research [ 37 ] can explain why more mixed assessments are being introduced in the EU.…”