Toward on-sky adaptive optics control using reinforcement learning

Abstract

Context. The direct imaging of potentially habitable exoplanets is one prime science case for the next generation of high contrast imaging instruments on ground-based, extremely large telescopes. To reach this demanding science goal, the instruments are equipped with eXtreme Adaptive Optics (XAO) systems which will control thousands of actuators at a framerate of kilohertz to several kilohertz. Most of the habitable exoplanets are located at small angular separations from their host stars, where the current control laws of XAO systems leave strong residuals. Aims. Current AO control strategies such as static matrix-based wavefront reconstruction and integrator control suffer from a temporal delay error and are sensitive to mis-registration, that is, to dynamic variations of the control system geometry. We aim to produce control methods that cope with these limitations, provide a significantly improved AO correction, and, therefore, reduce the residual flux in the coronagraphic point spread function (PSF). Methods. We extend previous work in reinforcement learning for AO. The improved method, called the Policy Optimization for Adaptive Optics (PO4AO), learns a dynamics model and optimizes a control neural network, called a policy. We introduce the method and study it through numerical simulations of XAO with Pyramid wavefront sensor (PWFS) for the 8-m and 40-m telescope aperture cases. We further implemented PO4AO and carried out experiments in a laboratory environment using Magellan Adaptive Optics eXtreme system (MagAO-X) at the Steward laboratory.Results. PO4AO provides the desired performance by improving the coronagraphic contrast in numerical simulations by factors of 3-5 within the control region of deformable mirror and PWFS, both in simulation and in the laboratory. The presented method is also quick to train, that is, on timescales of typically 5-10 seconds, and the inference time is sufficiently small (< ms) to be used in real-time control for XAO with currently available hardware even for extremely large telescopes.

Polarization aberrations in next-generation giant segmented mirror telescopes (GSMTs)

“…It is being designed to support both blue and red channels expected to achieve a postprocessed contrast of 10 −8 at 15 mas and 10 −9 at 100 mas angular separation from the star . The required inner working angle (IWA) of 15 mas is between 1 and 3λ/D where a raw contrast of 10 −5 to 10 −4 is expected Nousiainen et al 2022).…”

Tempestas ex machina: a review of machine learning methods for wavefront control

“…Haffert, 2021 26 argues that running in closed-loop maintains the condition of linearity for both Pyramid wavefront sensors (PWFS) and DM actuators, meaning non-linear reconstructions and system dynamics are not worth encapsulating in temporally evolving models. However, other works 36,39 demonstrate that a non-linear solver can learn the entire system model, including wavefront sensing response, deformable mirror dynamics, and DM to wavefront sensor misregistration, which leads to a more painless and flexible system implementation. Furthermore, Wong 2021 21 found that despite converging to the same solution, the neural network found the answer with ∼5 seconds of training data, as compared to a linear predictor which needed ∼40 seconds of training data.…”

Visible extreme adaptive optics on extremely large telescopes: towards detecting oxygen in Proxima Centauri b and analogs

“…28 Perhaps more importantly however, adaptive predictive control 29 will allow a system to constantly adapt its controller to provide optimal control. [30][31][32] This is important as the conditions during observations are constantly changing. 29…”