Content validation of a measure of R&D effectiveness

Abstract

The overall purpose of this paper is to analyse the content validity of a tool for measuring research and development (R&D) effectiveness in industry using an approach known as scale or construct validation.Since a large number of indicators is needed in order to measure this concept, and it is often difficult to find qualitative measures that would provide more information than quantitative measures or purely numerical magnitudes, we have constructed a scale that enables us to create a multiindicator to measure R&D inputs, processes, outputs and results. This multiindicator also enables us to group together all the relevant data obtained from the R&D management literature, which we then validate by consulting the opinion of experts from two firms that are very active in R&D and we have consulted two more nationally recognized Spanish researchers on R&D.

The implications of R & D management for organisational effectiveness: A literature review

“…For this lens, the portfolio of 25 articles is divided into eight different groups: (i) articles that do not contain information, (ii) articles that recognise the importance of integration but do not show how to do it, (iii) articles that make integration with correlation analysis between variables, (iv) articles that assign weights to each criterion, (v) articles that use the BSC cause-effects maps, (vi) articles that are made with the ANP (Analytic Network Process), (vii) articles with trade-off analysis usage, and (viii) articles that use R & D processes. Some 32% of selected articles do not understand or contain information about integration [2,18,42,45,51,53,54,56], and 68% do recognise the importance of integrating indicators. From this second group, three do not make explicit how they intend to integrate scales [43,47,55].…”

“…It is apparent that 32% of the articles do not present processes to measure the relevant aspects [2,18,19,36,39,44,45,56].…”

“…In the second set, there are seven articles that show a process identifying the relevant aspects and metrics to evaluate R & D management process and performance based on a literature review [2,18,19,36,40,45,51].…”

“…Ten articles in the portfolio examine the performance of new product development [18,37,38,42,47,49, 51±54], six assess the project management in the context of R & D [41,43,44,50,55,57], two evaluate innovation management [2,46], one focuses on knowledge management [39], and the remaining six articles assess the process of management of R & D [19,36,40,45,48,56]. There is a consensus amongst the articles that the ultimate goal of evaluation is to improve the performance of the considered context.…”

“…11 Content validation of a measure of R & D effectiveness [45]. 12 A balanced scorecard framework for R & D [19].…”