The impact of filtering self‐organizing maps: a case study with Australian pressure and rainfall

Abstract

This study presents a semi‐objective set of criteria for filtering the time series of a self‐organizing map (SOM, a method for collapsing a complex data set onto a series of typical instances/nodes). The purpose of the filter (or selection process) is to avoid the problem of associating anomalous time steps with SOM nodes that do not adequately describe that time step. The filter is based on measures of the match of the original field to the nearest SOM node. We demonstrate the method by using 21 years of MSLP reanalysis data for the Australian region from the European Centre for Medium Range Weather Forecasting product, ERA‐Interim. We show that there are instances in which the SOM is incapable of describing the original data and that these instances can be removed using a set of criteria. We then show that this filtering/removal process improves both the quantitative and qualitative matching of the SOM to the MSLP data. Finally, we show that SOM‐associated precipitation at three Australian cities (Melbourne, Sydney and Perth) is significantly influenced by the MSLP distributions that do not relate to the SOM and were removed by the filtering. The results show that care is required when interpreting trends inferred from data sorted into SOM nodes.

On the use of self‐organizing maps for studying climate extremes

“…Indeed, when using SOMs for studying extremes, if the highly generalized SOM patterns are not able to realistically characterize the synoptic conditions associated with the extreme event, then the relevance of the SOM patterns could be questionable. Huva et al [] demonstrate that, without careful treatment, the generalized SOM nodes often do not adequately represent the relevant synoptic circulation features associated with rainfall at major Australian cities and this may lead to erroneous conclusions regarding how circulation patterns have contributed to rainfall trends at these locations. Lennard and Hegerl [] linked rainfall trends to SOM patterns but also cautioned that the SOM patterns used in their study do not capture closed low‐pressure systems known to be linked to heavy rainfall in that region.…”

Influential synoptic weather types for a future renewable energy dependent national electricity market

“…Each time step from 2010-2011 was assigned a SOM node based on Euclidean distance and then the filtering process outline in Huva et al (2015) was employed (using the (20 hPa, 5 %, 4 hPa) set of conditions). In total there were 177 false assignments out of 2,920 ERA-Interim time steps for 2010-2011.…”

“…After training the SOM it is then possible to go through the original data and assign the closest looking SOM node (known as the Best Matching Unit/SOM node (BMU)) to each member of the data set (determined using Euclidean distance). Following the mapping of the SOM nodes back on to the original data the filtering process outlined in Huva et al (2015) was also undertaken. The process of checking the BMUs ensures that the purpose of the current study--to determine the synoptic influences on wind and solar output--is not undermined by falsely assigning SOM nodes to conditions from the original data-set.…”

“…It also raises the question as to what the large-scale synoptic influences are at these times of low output, and if any of these low RE episodes have significant links to particular types of weather events. Creating and filtering the continental SOM By testing SOMs of various sizes it was determine that the appropriate number of nodes was 30 (using the technique described in Huva et al 2015 to balance data representation and aesthetics). The SOM with 30 nodes that had the closest match to the original 1989-2009 training data had initial alpha of 0.5 and is shown in Figure 6.…”

Recursive multivariate principal‐monotonicity inferential climate downscaling

“…A growing number of computational learning algorithms including tree-based methods (Goyal et al, 2012), genetic programming (Pour et al, 2014), support vector machines (Tripathi et al, 2006), and relevance vector machines (Ghosh and Mujumdar, 2008) were developed as well. Classification methods that were exploited based on expert systems (Dong et al, 2011(Dong et al, , 2012(Dong et al, , 2013(Dong et al, , 2014c, fuzzy rules (Bárdossy et al, 2002;Khan and Valeo, 2016), stepwise cluster analysis (Wang et al, 2013), and selforganizing maps (Huva et al, 2015) were widely applied within a context of downscaling. However, few studies were dedicated to incorporating all of the aforementioned complexities into the downscaling process without unreasonable simplifications through proposition of an effective downscaling approach.…”