Benchmarking health technology assessment agencies—methodological challenges and recommendations

Abstract

Objectives The objectives of the study were to establish a benchmarking tool to collect metrics to enable increased clarity regarding the differences and similarities across health technology assessment (HTA) agencies, to assess performance within and across HTA agencies, identify areas in the HTA processes in which time is spent and to enable ongoing performance improvement. Methods Common steps and milestones in the HTA process were identified for meaningful benchmarking among agencies. A benchmarking tool consisting of eighty-six questions providing information on HTA agency organizational aspects and information on individual new medicine review timelines and outcomes was developed with the input of HTA agencies and validated in a pilot study. Data on 109 HTA reviews from five HTA agencies were analyzed to demonstrate the utility of this tool. Results This study developed an HTA benchmarking methodology, comparative metrics showed considerable differences among the median timelines from assessment and appraisal to final HTA recommendation for the five agencies included in this analysis; these results were interpreted in conjunction with agency characteristics. Conclusions It is feasible to find consensus among HTA agencies regarding the common milestones of the review process to map jurisdiction-specific processes against agreed metrics. Data on characteristics of agencies such as their scope and remit enabled results to be interpreted in the appropriate local context. This benchmarking tool has promising potential utility to improve the transparency of the review process and to facilitate both quality assurance and performance improvement in HTA agencies.

An Integrated Framework to Measure the Performance of Inter-Organizational Programme on Health Technology Assessment

“…Finally, the dissemination phase, which includes monitoring and education, can be analyzed with another specific set of ad-hoc indicators. Learning is often indirect and implicit in multi-organizational relationships [53], however in this case learning is not a sub product of collaboration but is strategic and directly related to the reason for developing the HTA network programmes [37,72] Indeed, both central or local levels are responsible for disseminating the activities to align knowledge and experiences across network actors, as well as the monitoring of the application of the recommendations and the results of this process in the real world. Information sharing is often reported as a main driver of the initial decision by an organization to participate in a (voluntary/partnership) network, and a mean for developing deeper collaborations and align member-value proposition (mission specific goals) [13].…”