Generalized Inverse Optimization through Online Learning

Abstract

Inverse optimization is a powerful paradigm for learning preferences and restrictions that explain the behavior of a decision maker, based on a set of external signal and the corresponding decision pairs. However, most inverse optimization algorithms are designed specifically in batch setting, where all the data is available in advance. As a consequence, there has been rare use of these methods in an online setting suitable for real-time applications. In this paper, we propose a general framework for inverse optimization through online learning. Specifically, we develop an online learning algorithm that uses an implicit update rule which can handle noisy data. Moreover, under additional regularity assumptions in terms of the data and the model, we prove that our algorithm converges at a rate of O(1/ √ T ) and is statistically consistent. In our experiments, we show the online learning approach can learn the parameters with great accuracy and is very robust to noises, and achieves a dramatic improvement in computational efficacy over the batch learning approach.

Inverse Bayesian Optimization: Learning Human Acquisition Functions in an Exploration vs Exploitation Search Task

“…The estimand in our project-the acquisition function of a Bayesian optimization-is analogous to the risk preferences estimated in Li's paper. Additionally, the sequential nature of the learning problem we study relates to online learning in inverse optimization (Bärmann et al, 2018;Dong et al, 2018;Dong and Zeng, 2020) and inverse Markov decision processes (Erkin et al, 2010).…”