Abstract-Ant Colony Optimization (ACO) has proven to be a very powerful optimization heuristic for combinatorial optimization problems. This paper introduces a new type of ACO algorithm that will be used for routing along multiple routes in a network as opposed to optimizing a single route. Contrary to traditional routing algorithms, the Ant Dispersion Routing (ADR) algorithm has the objective of determining recommended routes for every driver in the network, in order to increase network efficiency. We present the framework for the new ADR algorithm, as well as the design of a new cost function that translates the motivations and objectives of the algorithm. The proposed approach is illustrated with a small simulationbased case study for the Singapore Expressway Network.

Ant Colony Optimization (ACO) has proven to be a very powerful optimization heuristic for Combinatorial Optimization Problems. While being very successful for various NP-complete optimization problems, ACO is not trivially applicable to control problems. In this paper a novel ACO algorithm is introduced for the automated design of optimal control policies for continuous-state dynamic systems. The so called Fuzzy ACO algorithm integrates the multi-agent optimization heuristic of ACO with a fuzzy partitioning of the state space of the system. A simulated control problem is presented to demonstrate the functioning of the proposed algorithm.

No abstract

Ant Colony Optimization (ACO) is an optimization heuristic for solving combinatorial optimization problems and it is inspired by the swarming behavior of foraging ants. ACO has been successfully applied in various domains, such as routing and scheduling. In particular, the agents, called ants here, are very efficient at sampling the problem space and quickly finding good solutions. Motivated by the advantages of ACO in combinatorial optimization, we develop a novel framework for finding optimal control policies that we call Ant Colony Learning (ACL). In ACL, the ants all work together to collectively learn optimal control policies for any given control problem for a system with nonlinear dynamics. In this chapter, we will discuss the ACL framework and its implementation with crisp and fuzzy partitioning of the state space. We demonstrate the use of both versions in the control problem of two-dimensional navigation in an environment with variable damping and discuss their performance.

http://www.dcsc.tudelft.nl/˜bdeschutterPurpose -In this paper, a novel Ant Colony Optimization (ACO) approach to optimal control is proposed. The standard ACO algorithms have proven to be very powerful optimization metaheuristic for combinatorial optimization problems. They have been demonstrated to work well when applied to various NP-complete problems, such as the traveling salesman problem. In this paper, ACO is reformulated as a model-free learning algorithm and its properties are discussed. Design/methodology/approach -First, it is described how quantizing the state space of a dynamic system introduces stochasticity in the state transitions and transforms the optimal control problem into a stochastic combinatorial optimization problem, motivating the ACO approach. The algorithm is presented and is applied to the time-optimal swing-up and stabilization of an underactuated pendulum. In particular, the effect of different numbers of ants on the performance of the algorithm is studied. Findings -The simulations show that the algorithm finds good control policies reasonably fast. An increasing number of ants results in increasingly better policies. The simulations also show that although the policy converges, the ants keep on exploring the state space thereby capable of adapting to variations in the system dynamics. Research limitations/implications -This research introduces a novel ACO approach to optimal control and as such marks the starting point for more research of its properties. In particular, quantization issues must be studied in relation to the performance of the algorithm. Originality/value -The work presented is original as it presents the first application of ACO to optimal control problems.

No abstract

In this paper, we study the convergence of the pheromone levels of Ant Colony Learning (ACL) in the setting of discrete state spaces and noiseless state transitions. ACL is a multi-agent approach for learning control policies that combines some of the principles found in ant colony optimization and reinforcement learning. Convergence of the pheromone levels in expected value is a necessary requirement for the convergence of the learning process to optimal control policies. In this paper, we derive upper and lower bounds for the pheromone levels and relate those to the learning parameters and the number of ants used in the algorithm. We also derive upper and lower bounds on the expected value of the pheromone levels.

Abstract-In this paper, we discuss the Ant Colony Learning (ACL) paradigm for non-linear systems with continuous state spaces. ACL is a novel control policy learning methodology, based on Ant Colony Optimization. In ACL, a collection of agents, called ants, jointly interact with the system at hand in order to find the optimal mapping between states and actions. Through the stigmergic interaction by pheromones, the ants are guided by each others experience towards better control policies. In order to deal with continuous state spaces, we generalize the concept of pheromones and the local and global pheromone update rules. As a result of this generalization, we can integrate both crisp and fuzzy partitioning of the state space into the ACL framework. We compare the performance of ACL with these two partitioning methods by applying it to the control problem of swinging-up and stabilizing an under-actuated pendulum.