Exact Maximum Entropy Inverse Optimal Control for modeling human attention switching and control

Abstract

Maximum Causal Entropy (MCE) Inverse Optimal Control (IOC) has become an effective tool for modeling human behavior in many control tasks. Its advantage over classic techniques for estimating human policies is the transferability of the inferred objectives: Behavior can be predicted in variations of the control task by policy computation using a relaxed optimality criterion. However, exact policy inference is often computationally intractable in control problems with imperfect state observation. In this work, we present a model class that allows modeling human control of two tasks of which only one be perfectly observed at a time requiring attention switching. We show how efficient and exact objective and policy inference via MCE can be conducted for these control problems. Both MCE-IOC and Maximum Causal Likelihood (MCL)-IOC, a variant of the original MCE approach, as well as Direct Policy Estimation (DPE) are evaluated using simulated and real behavioral data. Prediction error and generalization over changes in the control process are both considered in the evaluation. The results show a clear advantage of both IOC methods over DPE, especially in the transfer over variation of the control process. MCE and MCL performed similar when training on a large set of simulated data, but differed significantly on small sets and real data.

Memory Pattern Identification for Feedback Tracking Control in Human–Machine Systems

“…Many other different approaches are being investigated in academia to reproduce biofidelic manual control, particularly in the field of highway driving. These range from optimal control (Schmitt, Bieg, Manstetten, Herman, & Stiefelhagen, 2016) and inverse optimal control (Inga, Eitel, Flad, & Hohmann, 2018) to multiplicative models (Martínez-García & Gordon, 2018).…”

I See What You See: Inferring Sensor and Policy Models of Human Real-World Motor Behavior

“…In addition to that, we derive a concrete implementation of the concept in the problem class of Sensor Scheduling LQGs (SLQG)s. In contrast to ordinary LQGs (Phatak et al 1976;Golub, Chase, and Byron 2013;Chen and Ziebart 2015) that are characterized by a single static sensor model, SLQGs allow control of the sensor model for active information gathering. Thereby, we both extend the implementation of SERD for small discrete and fully-observable problems (Herman et al 2016) to SLQGs and previous work on IOC in SLQGs (Schmitt et al 2016a) to inference of sensor models. Finally, we demonstrate the effectiveness of the proposed method on data of a new driving experiment conducted in real traffic.…”

“…In the remaining part of this work we will address the POMDP class of SLQGs. Building on the work of (Schmitt et al 2016a), it is shown that most parts of Eq. ( 2)-( 10) allow exact and tractable computation, while there exists an approximation technique and tractable special cases for the remaining ones.…”

“…, where the terms Ω1 t , Ω 1 t , ω2 t , ω 2 t , ω3 t , ω 3 t can be obtained in closed-form and independently of states x z t (Schmitt et al 2016a). Here, the expression [y; x] denotes the vertical concatenation of vectors x, y.…”

“…in ordinary LQGs (Chen and Ziebart 2015). Furthermore, also regularly enforcing switches to a model of perfect sensing of x p t , as in (Schmitt et al 2016a), can reduce the computational burden. This is because shorter sequences (x z t , x z t +1 , .…”