Fingerprint resampling: A generic method for efficient resampling

Abstract

In resampling methods, such as bootstrapping or cross validation, a very similar computational problem (usually an optimization procedure) is solved over and over again for a set of very similar data sets. If it is computationally burdensome to solve this computational problem once, the whole resampling method can become unfeasible. However, because the computational problems and data sets are so similar, the speed of the resampling method may be increased by taking advantage of these similarities in method and data. As a generic solution, we propose to learn the relation between the resampled data sets and their corresponding optima. Using this learned knowledge, we are then able to predict the optima associated with new resampled data sets. First, these predicted optima are used as starting values for the optimization process. Once the predictions become accurate enough, the optimization process may even be omitted completely, thereby greatly decreasing the computational burden. The suggested method is validated using two simple problems (where the results can be verified analytically) and two real-life problems (i.e., the bootstrap of a mixed model and a generalized extreme value distribution). The proposed method led on average to a tenfold increase in speed of the resampling method.

Synergized Bootstrapping: The Whole is Faster than the Sum of Its Parts

“…In this paper, we introduced a new method for speeding up the optimization procedures in the context of re-sampling methods and we applied the method to the non-parametric bootstrapping of the parameters of a univariate two-component mixture model. Just like the fingerprint method (Mestdagh et al, 2015), the proposed synergized bootstrap method exploits the fact that the characteristics of the re-sampled datasets are very similar, and so are the related optimization problems. Moreover, it relies on the fact that, for a large class of cost functions, the costs of a parameter set for the different cost functions can be obtained at a very low computing cost.…”

“…Notwithstanding the present-day computing power, a main disadvantage of re-sampling methods is still their computational burden (Mestdagh, Verdonck, Duisters, & Tuerlinckx, 2015). This burden is conspicuously present when iterative methods are required to maximize a likelihood or minimize a cost function for every re-sampled dataset.…”

“…Therefore, when fitting the model to the re-sampled datasets, a similar task has to be executed over and over again. The fingerprint re-sampling method (Mestdagh et al, 2015) exploits this feature by learning the relation between data characteristics and the corresponding model estimates. Once this relation has been established, it can be used to suggest more appropriate starting points for the iterative estimation process.…”

“…If the mapping between the data space and the model space is non-smooth or has discontinuities, the applicability of the fingerprint method can be called into question. The method has in fact only been tested for smooth, convex min-log-likelihood functions using local optimizers (Mestdagh et al, 2015). For such min-log-likelihood functions, the estimates of the re-sampled datasets typically lie closely together in the parameter space and the relation between data characteristics and estimates can nicely be established.…”

Prepaid parameter estimation without likelihoods

“…Third (panel B2), interpolation methods are used to find the relation s = f ( θ ) between the parameter values and the summary statistics for the selected points of the previous step [10, 11]. In this paper, we use tuned least squares support vector machines, LS-SVM [12].…”