Data-Driven Public R&D Project Performance Evaluation: Results from China

Abstract

In public R&D projects, to improve the decision-making process and ensure the sustainability of public investment, it is indispensable to effectively evaluate the project performance. Currently, public R&D project management departments and various academic databases have accumulated a large number of project-related data. In view of this, we propose a data-driven performance evaluation framework for public R&D projects. In our framework, we collect structured and unstructured data related to completed projects from multiple websites. Then, these data are cleaned and fused to form a unified dataset. We train a project performance evaluation model by extracting the project performance information implicit in the dataset based on multi-classification supervised learning algorithms. When facing a new project that needs to be evaluated, its performance can be automatically predicted by inputting the characteristic information of the project into our performance evaluation model. Our framework is validated based on the project data of the National Natural Science Foundation of China (NSFC) in terms of four performance measures (i.e., Accuracy, Recall, Precision, F1 score). In addition, we provide a case study that applies our framework to evaluate the project performance in the logistics and supply chain area of NSFC. In conclusion, this paper contributes to the body of knowledge in sustainability by developing a data-driven method that equips the decision-maker with an automated project performance evaluation tool to make sustainable project decisions.

Machine Learning Applied to Logistics Decision Making: Improvements to the Soybean Seed Classification Process

“…In the following phase, machine learning models were trained on the training sets and subsequently tested. The algorithms considered included Linear Regression [54], Random Forest [55], Gradient Boosting [56], and Decision Tree [30] as they are well applied in the literature [57,58]. Model selection was based on the root mean squared error (RMSE), a metric that offers a comprehensive assessment of prediction accuracy, assigning equal importance to both small and large errors.…”