High-speed detection of emergent market clustering via an unsupervised parallel genetic algorithm

Abstract

We implement a master-slave parallel genetic algorithm (PGA) with a bespoke log-likelihood fitness function to identify emergent clusters within price evolutions. We use graphics processing units (GPUs) to implement a PGA and visualise the results using disjoint minimal spanning trees (MSTs). We demonstrate that our GPU PGA, implemented on a commercially available general purpose GPU, is able to recover stock clusters in sub-second speed, based on a subset of stocks in the South African market. This represents a pragmatic choice for low-cost, scalable parallel computing and is significantly faster than a prototype serial implementation in an optimised C-based fourth-generation programming language, although the results are not directly comparable due to compiler differences. Combined with fast online intraday correlation matrix estimation from high frequency data for cluster identification, the proposed implementation offers cost-effective, near-real-time risk assessment for financial practitioners.

Fast Super-Paramagnetic Clustering

“…The method proposed in [5,6,15] allows for all sorts of mutations but can be sensitive to initial conditions: At every step a new generation of individuals is mutated, evaluated, and a group of the best candidates survives until the next algorithm's iteration. It has its disadvantages which are discussed in Table I: I1 Convergence Criteria: Assuming the existence of multiple local maxima it tries to navigate around these "sub-optimal" solutions on its way to a global maximum.…”

“…We call the new algorithm Agglomerative Super-Paramagnetic Clustering (ASPC) and it has the benefit of being less computationally expensive than the PGAs implemented in [5,6,15]. Here the key insight leading to the performance enhancement arises from being able serialize the algorithm into a brute-force search across * Electronic address: ylblio001@myuct.ac.za † Electronic address: tim.gebbie@uct.ac.za 1 For a review on the last 20 years on financial market correlation based data clustering see [9], and more generally on data clustering see [7].…”

Learning zero-cost portfolio selection with pattern matching

“…pattern-matching. Clusters can be chosen by a variety of methods, we would like to promote two methods: (i) correlation matrix based methods [ 27 ], and (ii) clusters based on economic classifications of stocks (for example, using ICB (Industry Classification Benchmark) sectors classifications [ 28 ]). The prior method, correlation based methods, have outputs that can be directly used as inputs into the algorithms discussed here, specifically via s ( n ), the cluster membership parameters.…”

Using real-time cluster configurations of streaming asynchronous features as online state descriptors in financial markets

“…is the intra-cluster correlation. Hendricks et al (2016b) show that the likelihood function specified in Equation 4 can be used as an objective function in a high-speed, scalable parallel genetic algorithm (PGA), where candidate cluster configurations are evaluated and successively improved until a configuration best explains the inherent structure suggested by a correlation matrix. We will utilise their computational solution, since it provides the near-realtime efficiency required for our proposed online algorithm.…”