Evaluating the Information Value for Measures of Systemic Conditions

Abstract

Timely identifi cation of coincident systemic conditions and forward-looking capacity to anticipate adverse developments are critical for macroprudential policy. Despite clear recognition of these factors in literature, an evaluation methodology and empirical tests for the information value of coincident measures are lacking. This paper provides a twofold contribution to the literature: (i) a general-purpose evaluation framework for assessing information value for measures of systemic conditions, and (ii) an empirical assessment of the information value for several alternative measures of US systemic conditions. We fi nd substantial differences among the measures, of which the Cleveland Financial Stress Index shows best-in-class identifi cation performance. In terms of forecasting performance, Kamakura's Troubled Company Index, Cleveland Financial Stress Index, and Goldman Sachs Financial Conditions Index show moderately stable usefulness metrics over time.

The Financial Stress Index: Identification of Systemic Risk Conditions

“…To evaluate the candidate weighting schemes we follow the work of Oet et al [60] which develops a framework to evaluate the information value of measures describing the state of the US financial system. They suggest comparing each FSI against a benchmark of financial system crisis using the Type I error, Type II error, noise to signal ratio ( ), information value ( ), and relative usefulness ( (µ)) metrics.…”

“…While volatility series may not offer insight into the drivers of stress, they are widely used to provide a general overview of market conditions. Oet et al [60] construct a benchmark of stress from six volatility series representing different segments of the financial system (shown in Figure 1). 6 We convert the volatility series into a binary indicator of systemic crisis following Equations (21) and (22).…”

“…Other FSIs typically introduce volatility measures which offer less intuition about the specific causes of stress or potential remedial action. Aside from providing a methodology for determining the quality of information provided by a financial stress measure, [60] compare a collection of US FSIs. They find that while most measures considered possess attractive noise to signal ratios, CFSI broadly outperformed alternative stress measures in terms of relative usefulness across all frequencies.…”

Supervising System Stress in Multiple Markets

“…To determine the quality of information provided by coincident measures of systemic conditions, Oet et al [21] consider a combination set of metrics of association between a candidate measure and the benchmark, including Type I error rate, Type II error rate, information value ( ), noise to signal ( ), and relative usefulness ( (µ) ). Applying this empirical evidence, we suggest that the acceptance criteria for a candidate stress measure include the following information quality metrics: (1) noise to signal ratio less than 0.3 (a ratio less than 1 implies a beneficial measure), (2) information value between 0.3 and 0.6 (Siddiqi [22] describes 0.3 to 0.5 as a strong), and (3) relative usefulness of at least 0.3.…”

“…We calculate currency crashes and covered interest spreads for seven of the G20 economies with floating exchange rates and large trade balances with the U.S. 3 This generalizes the ad hoc alternatives of crisis lists and expert surveys. 4 Section 2.4 summarizes the benchmark construction (for full details see [21] and are the values of all U.S. mortgagerelated securities outstanding from the "US MortgageRelated Issuance and Outstanding" file ("US Agency MBS Outstanding" and "NonAgency Outstanding" worksheets).…”