Active anomaly detection for time-domain discoveries

Abstract

Aims. We present the first piece of evidence that adaptive learning techniques can boost the discovery of unusual objects within astronomical light curve data sets. Methods. Our method follows an active learning strategy where the learning algorithm chooses objects that can potentially improve the learner if additional information about them is provided. This new information is subsequently used to update the machine learning model, allowing its accuracy to evolve with each new piece of information. For the case of anomaly detection, the algorithm aims to maximize the number of scientifically interesting anomalies presented to the expert by slightly modifying the weights of a traditional isolation forest (IF) at each iteration. In order to demonstrate the potential of such techniques, we apply the Active Anomaly Discovery algorithm to two data sets: simulated light curves from the Photometric LSST Astronomical Time-series Classification Challenge (PLAsTiCC) and real light curves from the Open Supernova Catalog. We compare the Active Anomaly Discovery results to those of a static IF. For both methods, we performed a detailed analysis for all objects with the ∼2% highest anomaly scores. Results. We show that, in the real data scenario, Active Anomaly Discovery was able to identify ∼80% more true anomalies than the IF. This result is the first piece of evidence that active anomaly detection algorithms can play a central role in the search for new physics in the era of large-scale sky surveys.

ParSNIP: Generative Models of Transient Light Curves with Physics-Enabled Deep Learning

“…In practice, the detection of novel transients should involve a feedback loop where resources are dedicated to followup candidates, and where these are then added back into the training set. This form of "active learning" is discussed in detail in Ishida et al (2019) and an application to isolation forests for anomaly detection is discussed in Ishida et al (2021). The ParSNIP model is ideally suited to being used with these techniques.…”

Active deep learning method for the discovery of objects of interest in large spectroscopic surveys

“…Active learning has already shown to be successful in astronomy, for example, in estimating parameters of stellar population synthesis models by Solorio et al (2005) or for the classification of light curves of variable stars by Richards et al (2012). Gupta et al (2016) used active learning to learn a model for photometric data classification from spectroscopic data (the work was extended by Vilalta et al (2019)), and recently, active learning was used to minimise the number of required spectroscopically confirmed labels in preparing training sets for the photometric classification of supernova light curves by Ishida et al (2019a) and for active anomaly detection in light curves of supernovae by Ishida et al (2019b). Moreover, active deep learning has been successfully tested in remote sensing by Liu et al (2017), with further examples reviewed in Yang et al (2018).…”

SNAD Transient Miner: Finding Missed Transient Events in ZTF DR4 using k-D trees

“…The SNAD 2 team has been continuously working in the development of anomaly detection algorithms which are able to prove their efficiency in real data while incorporating domain knowledge in the machine learning model -thus tailoring it according to the scientific interest of the expert (e.g., Pruzhinskaya et al, 2019;Aleo et al, 2020;Malanchev et al, 2021;Ishida et al, 2021). In this work, we present a hybrid approach for mining transients in large astronomical datasets, specifically ZTF DR4; moreover, our methodology can also be applied to the nightly ZTF alert-stream via timedomain brokers like ANTARES (Matheson et al, 2021) and FINK (Möller et al, 2021).…”