Research on evaluation indices and calculation method of experimental design

Abstract

It is important to design the experiment for promoting efficiency and saving costs in engineering. With multiple design methods available, selecting a suitable experiment design is essential. However, evaluating different experimental designs poses a significant challenge. This article summarizes existing evaluation indices comprehensively and constructs an evaluation indices system of experiment design, offering a wide range of evaluation and comparison criteria. The evaluation indices system solves the problem of the inability to systematically compare different experimental designs. For indices that cannot be calculated directly, we propose three algorithms, namely, the Monte Carlo method, the coarse-fine granularity combination method, and the spatial discretization method, to get the approximation of these indices. Then, three different designs of an on-orbit reconnaissance satellite are compared by using the proposed evaluation indices and algorithms, including uniform design, Doptimal design, and sequential design. We employ a radar chart to combine multiple indices for comprehensive comparison and find that D-optimal design is the best choice. The feasibility of the experimental design evaluation index system is verified by case application. In addition, we use JMP to validate the correctness of the proposed method.