A Reinforcement Learning–Based Follow-up Framework

Abstract

Classification and characterization of variable and transient phenomena are critical for astrophysics and cosmology. Given the volume of nightly data produced by ongoing and future surveys such as LSST, it is critical to develop automatic tools that assist in observation decision-making, maximizing scientific output without resource wastage. We propose a reinforcement learning–based recommendation system for real-time astronomical observation of sources. We assess whether it is worth making further observations and recommend the best instrument from a preexisting candidate set of instruments. Current possible choices include single-band, multiband, and spectroscopic observations, although it is generalizable to any other kind of instrumentation. We rely on a reward metric to make recommendations, which incorporates the gain in a classification sense and the cost incurred for the queried observations. This metric is flexible and easily adaptable to different application scenarios. We run 24 simulations in an offline setting with preexisting observations from Gaia DR2 and SDSS DR14. We propose four comparison strategies, including the baseline strategy, which recommends based on the most similar past cases to the current case. Our strategy surpasses all other strategies in regard to reward. We reach an accuracy of 0.932, comparable to using the accuracy reached using all possible resources (0.948) but with half the number of photometric observations and 1000 times fewer spectroscopic resources. The baseline strategy lacks the complexity to achieve competitive results with our proposed strategy. Our framework is meant to aid continuous online observation decision-making and can be extended to incorporate multiple environmental and observation conditions.

Artificial Intelligence in Astronomical Optical Telescopes: Present Status and Future Perspectives

“…The SWO (Squeaky Wheel Optimization) optimizer based on the greedy algorithm is used in the SOFIA [76], Mars Rover, and THEMIS projects. Reinforcement learning is used for the planning of LSST telescopes and the ordering of sky areas observed by optical telescopes, improving the probability of optical telescopes discovering transient astronomical phenomena such as gravitational waves, gamma-ray bursts, and kilonovae [77,78].…”