In this study, a new approach based on adaptive dynamic programming (ADP) is proposed to control single-phase uninterruptible power supply inverters. The control scheme uses a single function approximator, called critic, to evaluate the optimal cost and determine the optimal switching. After offline training of the critic, which is a function of system states and elapsed time, the resulting optimal weights are used in online control, to get a smooth output AC voltage in a feedback form. Simulations show the desirable performance of this controller with linear and non-linear loads and its relative robustness to parameter uncertainty and disturbances. Furthermore, the proposed controller is upgraded so that the inverter is suitable for single-phase variable frequency drives. Finally, as one of the few studies in the field of ADP, the proposed controllers are implemented on a physical prototype to show the performance in practise.