Performance evaluation of metamodelling methods for engineering problems: towards a practitioner guide

Abstract

Metamodelling or surrogate modelling techniques are frequently used across the engineering disciplines in conjunction with expensive simulation models or physical experiments. With the proliferation of metamodeling techniques developed to provide enhanced performance for specific problems, and the wide availability of a diverse choice of tools in engineering software packages, the engineering task of selecting a robust metamodeling technique for practical problems is still a challenge. This research introduces a framework for describing the typology of engineering problems, in terms of dimensionality and complexity, and the modelling conditions, reflecting the noisiness of the signals and the affordability of sample sizes, and on this basis presents a systematic evaluation of the performance of frequently used metamodeling techniques. A set of metamodeling techniques, selected based on their reported use for engineering problems (i.e. Polynomial, Radial Basis Function, and Kriging), were systematically evaluated in terms of accuracy and robustness against a carefully assembled set of 18 test functions covering different types of problems, sampling conditions and noise conditions. A set of four real-world engineering case studies covering both computer simulation and physical experiments were also analysed as validation tests for the proposed guidelines. The main conclusions drawn from the study are that Kriging model with Matérn 5/2 correlation function performs consistently well across different problem types with smooth (i.e. not noisy) data, while Kriging model with Matérn 3/2 correlation function provides robust performance under noisy conditions, except for the very high noise conditions, where the Kriging model with nugget appears to provide better models. These results provide engineering practitioners with a guide for the choice of a metamodeling technique for problem types and modelling conditions represented in the study, whereas the evaluation framework and benchmarking problems set will be useful for researchers conducting similar studies. Structural and Multidisciplinary Optimization (2020) 61:159-186 Performance evaluation of metamodelling methods for engineering problems: towards a practitioner guide 161

Reliability and Reliability-based Sensitivity Analyses of Steel Moment-Resisting Frame Structure subjected to Extreme Actions

“…In recent years, meta-models are widely used among researchers over the world to reduce the computational costs of the reliability analyses by predicting the structural responses [33]. Meta-model techniques can provide an accurate representation of the PDF of the model response.…”

Gaussian Function On Response Surface Estimation

“…This enables us to identify the relationship between the input (predictors X) and output (response y) variables in a complex machine learning process, by fitting a global metamodel to the underlying model. The aim of [1] was to present a fair evaluation on several metamodels widely used by practitioners, comparing different characteristics of the techniques regarding robustness, accuracy, efficiency, etc. Given the fact that each metamodel is able to focus on one or two aspects of a model, they showed that not all techniques are powerful in every aspect, so the metamodeling strategy should be selected carefully.…”

On Selecting Composite Functions Based on Polynomials for Responses Describing Extreme Magnitudes of Structures

“…Polynomial responses are commonly applied [32,36,[44][45][46][47][48][49], and this idea follows some other applications in computational practice [50,51]. However, such approximations do not always provide satisfactory correctness [28,52]. Moreover, their application to describe the extreme magnitudes, for example, of structures seems unfortunate because polynomial limits at infinity are always infinite.…”