Detecting Location Shifts during Model Selection by Step-Indicator Saturation

Abstract

To capture location shifts in the context of model selection, we propose selecting significant step indicators from a saturating set added to the union of all of the candidate variables. The null retention frequency and approximate non-centrality of a selection test are derived using a 'split-half' analysis, the simplest specialization of a multiple-path block-search algorithm. Monte Carlo simulations, extended to sequential reduction, confirm the accuracy of nominal significance levels under the null and show retentions when location shifts occur, improving the non-null retention frequency compared to the corresponding impulse-indicator saturation (IIS)-based method and the lasso.

Evaluating Forecasts, Narratives and Policy Using a Test of Invariance

“…Hendry and Mizon (2011) demonstrate distortions in modelling from not handling outliers, and Castle et al (2016) show that outliers can even be distinguished from non-linear reactions as the latter hold at all observations whereas the former only occur at isolated points. Castle et al (2015) establish that the properties of step-indicator saturation are similar to IIS, but SIS exhibits higher power than IIS when location shifts occur. Ericsson and Reisman (2012) propose combining IIS and SIS in 'super saturation ' and Ericsson (2016) uses IIS to adjust an ex post mis-match between the truncated FMI Greenbook 'post-casts' and GDP outcomes.…”

“…Although SIS could be applied to (20) as a system, we will focus on its application to each equation separately. The gauge of step indicators in such individual marginal models is investigated in Castle et al (2015), who show that simulation-based distributions for SIS using Autometrics have a gauge, g, close to the nominal significance level α 1 . Section 4.1 notes their findings on the potency of SIS at the first stage to determine the occurrence of location shifts.…”

“…Here we propose a test of invariance based on step-indicator saturation (denoted SIS), following Castle et al (2015). SIS builds on the impulse-indicator saturation (IIS) approach proposed by Hendry et al (2008) to detect outliers and shifts in a model.…”

“…, T when the sample size is T, where 1 {t≤j} = 1 for observations up to j, and zero otherwise (so 1 {t≤T} is the intercept). Using a modified general-to-specific procedure, Castle et al (2015) establish the gauge and the null distribution of the resulting estimator of regression parameters. A two-block process is investigated analytically, where half the indicators are added and all significant indicators recorded, then that half is dropped, and the other half examined: finally, the two retained sets of indicators are combined.…”

“…Step-indicator saturation (SIS) is designed to detect location shifts of unknown magnitude, location and frequency at any point in the sample, see Castle et al (2015). Super exogeneity, which is required for policy validity, can be tested by checking whether any significant location shifts in models of the policy instruments are significant in the model of the target variables.…”