Bayesian Online Learning of the Hazard Rate in Change-Point Problems

Abstract

Change-point models are generative models of time-varying data in which the underlying generative parameters undergo discontinuous changes at different points in time known as change points. Changepoints often represent important events in the underlying processes, like a change in brain state reflected in EEG data or a change in the value of a company reflected in its stock price. However, change-points can be difficult to identify in noisy data streams. Previous attempts to identify change-points online using Bayesian inference relied on specifying in advance the rate at which they occur, called the hazard rate (h). This approach leads to predictions that can depend strongly on the choice of h and is unable to deal optimally with systems in which h is not constant in time. In this letter, we overcome these limitations by developing a hierarchical extension to earlier models. This approach allows h itself to be inferred from the data, which in turn helps to identify when change-points occur. We show that our approach can effectively identify change-points in both toy and real data sets with complex hazard rates and how it can be used as an ideal-observermodel for human and animal behavior when faced with rapidly changing inputs. Change-point models are generative models of time-varying data in which the underlying generative parameters undergo discontinuous changes at different points in time known as change points. Changepoints often represent important events in the underlying processes, like a change in brain state reflected in EEG data or a change in the value of a company reflected in its stock price. However, change-points can be difficult to identify in noisy data streams. Previous attempts to identify change-points online using Bayesian inference relied on specifying in advance the rate at which they occur, called the hazard rate (h). This approach leads to predictions that can depend strongly on the choice of h and is unable to deal optimally with systems in which h is not constant in time. In this letter, we overcome these limitations by developing a hierarchical extension to earlier models. This approach allows h itself to be inferred from the data, which in turn helps to identify when change-points occur. We show that our approach can effectively identify change-points in both toy and real data sets with complex hazard rates and how it can be used as an ideal-observer model for human and animal behavior when faced with rapidly changing inputs. Disciplines Medicine and Health Sciences

Random walk of motor planning in task-irrelevant dimensions

“…This algorithm has to know the hazard rate (the rate at which change points occur), but we obviously did not know this rate in advance. We therefore first ran the algorithm of Wilson et al (2010), which is an extension of the Adams and MacKay (2007) algorithm, to estimate the hazard rate. For both algorithms, we assumed that the generative distribution between change points was a Gaussian with unknown mean and known SD, which we derived from the data of the task-relevant component.…”

“…We further assumed that the prior, for the mean of the Gaussian, was a Gaussian itself with a mean of zero (the middle of the line) and a SD of 10ϫ the SD of the generative Gaussian, but the results did not depend on the exact value of this parameter. For the Wilson et al (2010) algorithm, we further assumed that the prior, for the hazard rate, followed a beta distribution with parameters ␣ ϭ 2, and ␤ ϭ 50, but the results were virtually independent of these values.…”

The perception of probability.

“…2005;Courville, Daw, Gordon, & Touretzky, 2004;Courville, Daw, & Touretzky, 2006;Dayan & Daw, 2008;Glimcher, 2003aGlimcher, , 2003bGlimcher, , 2009Montague et al, 1996;Nassar & Gold, 2013;Nassar et al, 2010Nassar et al, , 2012Seymour et al, 2004;Steyvers & Brown, 2006;Sugrue, Corrado, & New.some, 2005;Wilson et al, 2010). They have a long history (Fstes, 1957;Rescorla & Wagner, 1972).…”

Reward Processing in Motivational and Affective Disorders

“…Similarly, a good deal of decision-making neuroscience seeks to uncover how uncertainty is represented neurally (see Wilson et al, 2010;Nassar et al, 2012). A recent suggestion is that operating in an unstable environment is associated with tonic release (over a time course of minutes) of norepinephrine Dayan, 2003, 2005).…”