Randomized Shortest-Path Problems: Two Related Models

Abstract

This letter addresses the problem of designing the transition probabilities of a finite Markov chain (the policy) in order to minimize the expected cost for reaching a destination node from a source node while maintaining a fixed level of entropy spread throughout the network (the exploration). It is motivated by the following scenario. Suppose you have to route agents through a network in some optimal way, for instance by minimizing the total travel cost. Nothing particular up to now -you could use a standard shortest-path algorithm. Suppose, however, that you want to avoid pure deterministic routing policies in order, for instance, to allow some continual exploration of the network, to avoid congestion, or to avoid complete predictability of your routing strategy. In other words, you want to introduce some randomness/unpredictability in the routing policy, i.e., the routing policy is randomized. This problem, which will be called the randomized shortest-path problem (RSP), is investigated in this work. The global level of randomness of the routing policy is quantified by the expected Shannon entropy spread throughout the network, and is provided a priori by the designer. Then, necessary conditions allowing to compute the optimal randomized policy -minimizing the expected routing cost -are derived. Iterating these necessary conditions, reminiscent of Bellman's value iteration equations, allows to compute an optimal policy, that is, a set of transition probabilities in each node. Interestingly and surprisingly enough, this first model, while formulated in a totally different framework, is equivalent to Akamatsu's model (Akamatsu (1996)), appearing in transportation science, for a special choice of the entropy constraint. We therefore revisit Akamatsu's model by recasting it into a sum-over-paths statistical-physics formalism allowing to easily derive all the quantities of interest in an elegant, unified, way. For instance, it is shown that the unique optimal policy can be obtained by solving a simple linear system of equations. This second * Marco Saerens is also a Research Fellow of the IRIDIA Laboratory, Université Libre de Bruxelles. 1 model is therefore more convincing because of its computational efficiency and soundness. Finally, simulation results obtained on simple, illustrative, examples show that the models behave as expected.

Information Theory of Decisions and Actions

“…Since we determine the relevance of information by the value it allows the agent to achieve, we speak in this context also of valuable information 2 . A related, but less biologically and resourcemotivated approach is found in (Saerens et al 2009). The topic of the present paper is a significant generalization of these considerations to the full-fledged perceptionaction cycle, and their treatment in the context of a generalized Bellman-type recursion.…”

Asynchrony of Seasons: Genetic Differentiation Associated with Geographic Variation in Climatic Seasonality and Reproductive Phenology

“…To estimate the likelihood of dispersal between localities linked with genetic data, we considered three different dispersal models: linear paths, least-cost paths, and randomized shortest paths (McRae and Beier 2007;Saerens et al 2009). First, we calculated linear geographic distance, taking into account Earth's curvature.…”

The Effects of Human-Mediated Habitat Fragmentation on a Sedentary Woodland-Associated Species (Rhinolophus hipposideros) at Its Range Margin

“…• 'Randomised shortest paths' are an intermediate form between shortest paths and Brownian random walks, introduced by Saerens et al (2009).…”