Matrix Profile II: Exploiting a Novel Algorithm and GPUs to Break the One Hundred Million Barrier for Time Series Motifs and Joins

Abstract

Time series motifs have been in the literature for about fifteen years, but have only recently begun to receive significant attention in the research community. This is perhaps due to the growing realization that they implicitly offer solutions to a host of time series problems, including rule discovery, anomaly detection, density estimation, semantic segmentation, etc. Recent work has improved the scalability to the point where exact motifs can be computed on datasets with up to a million data points in tenable time. However, in some domains, for example seismology, there is an insatiable need to address even larger datasets. In this work we show that a combination of a novel algorithm and a high-performance GPU allows us to significantly improve the scalability of motif discovery. We demonstrate the scalability of our ideas by finding the full set of exact motifs on a dataset with one hundred million subsequences, by far the largest dataset ever mined for time series motifs. Furthermore, we demonstrate that our algorithm can produce actionable insights in seismology and other domains.

Parallel time series join using spark

“…The computational complexity of computing the matrix profile is O ( n 2 log n ). The authors also extended their work by using a GPU to accelerate the time series join, and the time complexity of computing matrix profile is reduced to O ( n 2 ) . Zhu et al proposed a new algorithm SCRIMP++ running on single machine and accelerated by GPU.…”

Time Series Chains: A Novel Tool for Time Series Data Mining

“…First, we need to find the left and right nearest neighbor of all the subsequences in the time series. We use an algorithm called LRSTOMP, a variant of the STOMP algorithm [Zhu et al, 2016] to compute such information. The time complexity of LRSTOMP is O(n 2 ) and the space complexity is O(n) (here n is the length of the time series), the same as STOMP [Zhu et al, 2016].…”

“…We use an algorithm called LRSTOMP, a variant of the STOMP algorithm [Zhu et al, 2016] to compute such information. The time complexity of LRSTOMP is O(n 2 ) and the space complexity is O(n) (here n is the length of the time series), the same as STOMP [Zhu et al, 2016]. Next, we find all the anchored chains within the time series based on Definition 2 (i.e., we "grow" chains from every subsequence in the time series).…”

“…Motif discovery has been used as a sub-routine in higher-level analytics, including classification, clustering, visualization [Hao et al, 2012], and rule-discovery [Shokoohi-Yekta et al, 2015]. Moreover, motif discovery has been applied to domains as diverse as severe weather prediction, robotics, medicine [Syed et al, 2010] and seismology [Zhu et al, 2016].…”

“…If a pattern is repeated (or conserved), there must be a latent system that occasionally produces the conserved behavior. For example, this system may be an overcaffeinated heart, sporadically introducing a motif pattern containing an extra beat (Atrial Premature Contraction [Lovallo et al, 2004]), or the system may be an earthquake fault, infrequently producing highly repeated seismograph traces because the local geology produces unique wave reflection/refractions [Zhu et al, 2016]. Time series motifs are a commonly used technique to gain insight into such latent systems, in essence, they can be seen as "generalizing the notion of a regulatory motif to operate robustly on non-genomic data" [Syed et al, 2010].…”