???This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. In most cases, these works may not be reposted without the explicit permission of the copyright holder." ???Copyright IEEE. Personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution to servers or lists, or to reuse any copyrighted component of this work in other works must be obtained from the IEEE.???Living organisms are under permanent pressure to take decisions with an impact on their success. Such decisions require information, which can be formulated in the precise sense of Shannon information. Since information processing is costly for organisms, this creates an adaptive pressure for cognition to be as informationally parsimonious as possible. Combining information theory with the theory of reinforcement learning for modeling tasks, we present a number of quantitative analyses how the cognitive burden of an agent deriving from a task can be relieved by the environment and, more specifically, its embodiment, i.e. how the agent "controller" is linked to the environment, via perception (in principle, but not further considered here) and action (this paper's main focus). The methodology presented offers a path towards a formal and quantitative treatment of Paul's and Pfeifer's concept of morphological computation in particular and their envisaged larger picture of offloading of computation onto the environment dynamics in general. In particular, it offers additional evidence for the central importance of the embodiment for the success of cognition