Detection of dispersed radio pulses: a machine learning approach to candidate identification and classification

Abstract

Searching for extraterrestrial, transient signals in astronomical data sets is an active area of current research. However, machine learning techniques are lacking in the literature concerning single-pulse detection. This paper presents a new, two-stage approach for identifying and classifying dispersed pulse groups (DPGs) in single-pulse search output. The first stage identified DPGs and extracted features to characterize them using a new peak identification algorithm which tracks sloping tendencies around local maxima in plots of signal-to-noise ratio vs. dispersion measure. The second stage used supervised machine learning to classify DPGs. We created four benchmark data sets: one unbalanced and three balanced versions using three different imbalance treatments. We empirically evaluated 48 classifiers by training and testing binary and multiclass versions of six machine learning algorithms on each of the four benchmark versions. While each classifier had advantages and disadvantages, all classifiers with imbalance treatments had higher recall values than those with unbalanced data, regardless of the machine learning algorithm used. Based on the benchmarking results, we selected a subset of classifiers to classify the full, unlabelled data set of over 1.5 million DPGs identified in 42,405 observations made by the Green Bank Telescope. Overall, the classifiers using a multiclass ensemble tree learner in combination with two oversampling imbalance treatments were the most efficient; they identified additional known pulsars not in the benchmark data set and provided six potential discoveries, with significantly less false positives than the other classifiers.

Scalable Solutions for Automated Single Pulse Identification and Classification in Radio Astronomy

“…For effective classification of very large data sets, algorithms must be time-efficient and scalable. This paper builds on our previous work, which focused on identifying and classifying transient radio signals received by large radio telescopes [10]. Transient radio signals are short bursts of radiation detected at radio frequencies from sources such as pulsars and rotating radio transients (RRATs).…”

“…D-RAPID offers three main contributions. First, we scale-up a modified version of the identification algorithm we first presented in [10] to run in parallel using Apache Spark on a Hadoop YARN distributed system and show that it outperforms its multithreaded counterpart by processing data up to five times faster. Second, we offer a novel automated multiclass pulsar classification technique, which improves the execution performance of RandomForest, an ensemble tree machine learning algorithm, by 47% with less than a 2% reduction in classification performance, and also more accurately classifies cases missed by binary classification.…”

“…As data sets become very large, candidate identification is increasingly processor intensive and becomes an execution performance bottleneck in the process. Our previous work focused on the identification and classification of dispersed pulse groups (DPGs) [10], and developing clustering algorithms to identify groups of single pulse events [24] in candidate plots generated by running PRESTO's single pulse processing tool, sin le_pulse_search.p .…”

Gaiaeclipsing binary and multiple systems. Supervised classification and self-organizing maps

“…Typing "machine learning astronomy" into the ADS search window brings up about 20 articles for the first five months of 2016 alone, on subjects as diverse as photometric and gamma-ray burst redshift estimation, detection of radio transients, glitches in gravitational wave detection and exoplanet science (e.g. Hoyle 2016; Devine et al 2016;Ukwatta et al 2016;Zevin & Gravity Spy 2016;Ford 2016).…”

Learn from every mistake! Hierarchical information combination in astronomy

“…Both subjects have recently seen the proliferation of methods used. For the first, there is nowadays an increasing variety of supervised classifiers; in the astronomical literature, Dubath et al (2011);Rimoldini et al (2012); Goldstein et al (2015); Devine et al (2016); Tramacere et al (2016) represent a few examples. The Gaia Variability Processing Pipeline (Eyer et al submitted) applies three methods for the classification of variable stars, Bayesian Networks, Gaussian Mixtures and Random Forest.…”