Learning-based control for a communicating mobile robot under unknown rates

Abstract

In problems such as surveying or monitoring remote regions, a mobile robot must transmit data over a wireless network with unknown, position-dependent transmission rates. We propose an algorithm to achieve this objective that learns approximations of the rate function and of an optimal-control solution that transmits the data in minimum time. The rates are estimated with supervised learning from the samples observed; and the control is found with dynamic programming sweeps around the current state of the robot that exploit the rate function estimate, combined with online reinforcement learning. For both synthetic and realistic rate functions, our experiments show that the learning algorithm empties the data buffer in less than twice the number of steps achieved by a model-based solution that requires to perfectly know the rate function.

Learning control for transmission and navigation with a mobile robot under unknown communication rates

“…Regarding the obstacles, it will assumed that there is a finite number of them and that the initial state is initialized sufficiently far away that the robot can still avoid the obstacle. Recall that in the version of PT solved in Buşoniu et al (2019), there were no obstacles, and the dynamics had to be first-order, without any signal y.…”

“…Note that in line 8, the rate function approximator R is used. Compared to the algorithm of Buşoniu et al (2019), this version eliminates two components that did not significantly contribute to performance: direct reinforcement learning and softmax exploration. Instead, here only greedy action selection is used, with optimistic initialization.…”

“…Compared to the preliminary conference paper (Buşoniu et al, 2019), fully novel contributions here are solving PN with unknown rates and the real-system results. Buşoniu et al (2019) focused only on a simpler version of PT, with first-order robot dynamics, synthetic rate functions, and without obstacles. For the PT version studied here, the robot dynamics are extended to be of arbitrary order, the method is illustrated with realistic rate functions, and -importantly -basic obstacle avoidance functionality is introduced.…”

“…Compared to the preliminary conference paper (Buşoniu et al, 2019), fully novel contributions here are solving PN with unknown rates and the real-system results. Buşoniu et al (2019) focused only on a simpler version of PT, with first-order robot dynamics, synthetic rate functions, and without obstacles.…”