Reinforcement Learning of Sequential Price Mechanisms

Abstract

We introduce the use of reinforcement learning for indirect mechanisms, working with the existing class of sequential price mechanisms, which generalizes both serial dictatorship and posted price mechanisms and essentially characterizes all strongly obviously strategyproof mechanisms. Learning an optimal mechanism within this class forms a partially-observable Markov decision process. We provide rigorous conditions for when this class of mechanisms is more powerful than simpler static mechanisms, for sufficiency or insufficiency of observation statistics for learning, and for the necessity of complex (deep) policies. We show that our approach can learn optimal or near-optimal mechanisms in several experimental settings.

Offsetting Unequal Competition through RL-assisted Incentive Schemes

“…[68] build a reinforcement learning model which achieves near-optimality subject to (exact) fairness or approximate-choice fairness. Recently, there is a series of works at the intersection of incentive-based mechanism design and reinforcement learning [69], [70], [71]. Among theoretical works, [70] investigate various theoretical aspects of the use of reinforcement learning for certain classes of indirect mechanisms whereas [71] propose incentive-aware PAC learning in the presence of strategic manipulation.…”

“…Recently, there is a series of works at the intersection of incentive-based mechanism design and reinforcement learning [69], [70], [71]. Among theoretical works, [70] investigate various theoretical aspects of the use of reinforcement learning for certain classes of indirect mechanisms whereas [71] propose incentive-aware PAC learning in the presence of strategic manipulation. Our work closely resembles [69] that build social planners for devising tax policies in dynamic economies for effectively balancing economic equality and productivity, where the agents are trained through deep reinforcement learning.…”