Anomaly Detection in the Open Supernova Catalog

Abstract

In the upcoming decade large astronomical surveys will discover millions of transients raising unprecedented data challenges in the process. Only the use of the machine learning algorithms can process such large data volumes. Most of the discovered transients will belong to the known classes of astronomical objects. However, it is expected that some transients will be rare or completely new events of unknown physical nature. The task of finding them can be framed as an anomaly detection problem. In this work, we perform for the first time an automated anomaly detection analysis in the photometric data of the Open Supernova Catalog (OSC), which serves as a proof of concept for the applicability of these methods to future large scale surveys. The analysis consists of the following steps: 1) data selection from the OSC and approximation of the pre-processed data with Gaussian processes, 2) dimensionality reduction, 3) searching for outliers with the use of the isolation forest algorithm, 4) expert analysis of the identified outliers. The pipeline returned 81 candidate anomalies, 27 (33%) of which were confirmed to be from astrophysically peculiar objects. Found anomalies correspond to a selected sample of 1.4% of the initial automatically identified data sample of ∼2000 objects. Among the identified outliers we recognised superluminous supernovae, non-classical Type Ia supernovae, unusual Type II supernovae, one active galactic nucleus and one binary microlensing event. We also found that 16 anomalies classified as supernovae in the literature are likely to be quasars or stars. Our proposed pipeline represents an effective strategy to guarantee we shall not overlook exciting new science hidden in the data we fought so hard to acquire. All code and products of this investigation are made publicly available‡.

Active anomaly detection for time-domain discoveries

“…The catalog is constantly evolving, but at any moment in time it is also known to contain some percentage of non-SN contaminants (Guillochon et al 2017;Pruzhinskaya et al 2019), which makes it well suited for our purposes. The real data analysis is based on the data set 4 first presented in Pruzhinskaya et al (2019). Therefore, the detailed description of quality cuts, the data selection process, and the preprocessing pipeline are given there.…”

“…This has led to the development of a number of machine learning (ML) algorithms for anomaly detection (AD) with a large range of applications (Mehrotra et al 2017). In astronomy, these techniques have largely been applied to areas such as the identification of anoma-lous galaxy spectra (Baron & Poznanski 2017), problematic objects in photometric redshift estimation tasks (Hoyle et al 2015), characterization of light curves (LCs) of transients (Zhang & Zou 2018;Pruzhinskaya et al 2019), and variable stars (e.g., Rebbapragada et al 2009;Nun et al 2014;Giles & Walkowicz 2019;Malanchev et al 2021), among others.…”

“…As a proof of concept, we applied the active anomaly detection (AAD) strategy proposed by Das et al (2017) to two different data sets: simulated LCs from the Photometric LSST Astronomical Time-series Classification Challenge 2 (PLAsTiCC; The PLAsTiCC team et al 2018) and real LCs from the Open Supernova Catalog 3 (OSC). Our goal with the real data analysis is to lower the burden inflicted by the ML algorithms on domain experts and propose a strategy that would improve the results presented in Pruzhinskaya et al 2019 while requiring the expert to confirm a lower number of sources. Used in combination with a traditional isolation forest (IF) algorithm, the method allows an increasingly large incidence of true positives (scientifically interesting anomalies) among the objects presented to the expert, in turn enabling a better allocation of resources with the evolution of a given survey.…”

“…All the feature extraction scenarios reported in Sect. 4 ofPruzhinskaya et al (2019) were tested. The one described here corresponds to the concatenation of the two feature extraction scenarios that presented a larger spread in anomaly scores(Pruzhinskaya et al 2019, Fig.…”

fink, a new generation of broker for the LSST community

“…The use of automated anomaly detection (AD) algorithms might help mitigate some of these issues. However, traditional AD techniques are known to deliver a high incidence of false positives (see e.g., Pruzhinskaya et al 2019). Taking into account that, in astronomy, further scrutiny of an anomaly candidate is only possible through expensive spectroscopic follow-up, this translates into a significant fraction of resources spent in non-interesting targets.…”