Process-conditioned investing with incomplete information using maximum causal entropy

Abstract

Abstract. Investing to optimally maximize the growth rate of wealth based on sequences of event outcomes has many information-theoretic interpretations. Namely, the mutual information characterizes the benefit of additional side information being available when making investment decisions [1] in settings where the probabilistic relationships between side information and event outcomes are known. Additionally, the relative variant of the principle of maximum entropy [2] provides the optimal investment allocation in the more general setting where the relationships between side information and event outcomes are only partially known [3]. In this paper, we build upon recent work characterizing the growth rates of investment in settings with inter-dependent side information and event outcome sequences [4]. We consider the extension to settings with inter-dependent event outcomes and side information where the probabilistic relationships between side information and event outcomes are only partially known. We introduce the principle of minimum relative causal entropy to obtain the optimal worst-case investment allocations for this setting. We present efficient algorithms for obtaining these investment allocations using convex optimization techniques and dynamic programming that illustrates a close connection to optimal control theory.

Shared autonomy via hindsight optimization for teleoperation and teaming

“…Ziebart (2010) showed that minimizing the worst-case predictive loss results in a model where the probability of a sequence decreases exponentially with cost, p ( ξ | g ) ∝ exp ( − C g normalu ( ξ ) ) . Importantly, one can efficiently learn a cost function consistent with this model from demonstrations (Ziebart et al, 2008).…”

The two kinds of free energy and the Bayesian revolution

“…For our one-step example, this results in the Bayes’ posterior It is straightforward to identify p des ( X ′) of Active Inference as a particular choice of a success probability p 0 ( R = 1| X ′), or equivalently, log p des ( X ′) as a reward function r = r ( X ′), so that the joint distribution (27) reduces to the reference function ϕ in (25) . Thus, the version of Active Inference in [ 101 ] is simply a variational formulation of Control as Inference that approximates exact posteriors of the form (28) , like other previous variational Bayes’ approaches [ 107 , 109 , 110 ].…”

“…Either an action distribution is built into the reference function, which presupposes optimal behavior by designing a value function Q that leads to desired consequences, or the outcome probability under the generative model p 0 is modified directly by multiplying p des to p 0 . The latter case is the variational version of Control as Inference, well-known in the machine learning literature [ 77 , 105 – 110 ]. Considering the issues of Q -value Active Inference discussed above, and the fact that Control as Inference does not rely on a desired distribution over outcomes, we could ask whether formulating preferences by assuming a desired distribution is well-advised.…”

Robust structure estimation of maximum causal entropy processes

“…In previous applications of the principle of maximum causal entropy [25], [26], [24], [29], the causal structure of the interacting processes was assumed. We investigate estimating this causal structure rather than assuming it in this work.…”

“…This corresponds to the L 0 norm of parameters associated with those constraints, shown in (24) With an appropriately chosen penalty weight, λ, the minimal structure that provides a feasible causally conditioned probability estimate would be optimal. Unfortunately, with α = 0, (24) can no longer be solved as a convex optimization problem. Both greedy approximations to the L 0 solution and relaxations to L 1 or L 2 penalties are instead possible.…”