Stochastic Optimal Control of Active Vehicle Suspensions Using Learning Automata

Abstract

This paper is concerned with the application of reinforcement learning to the stochastic optimal control of an idealized active vehicle suspension system. The use of learning automata in optimal control is a new application of this machine learning technique, and the principal aim of this work is to define and demonstrate the method in a relatively simple context, as well as to compare performance against results obtained from standard linear optimal control theory. The most distinctive feature of the approach is that no formal modelling is involved in the control system design; once implemented, learning takes place on-line, and the automaton improves its control performance with respect to a predefined cost function. An important new feature of the method is the use of subset actions, which enables the automaton to reduce the size of its action set at any particular instant, without imposing any global restrictions on the controller that is eventually learnt. The results, though based on simulation studies, suggest that there is great potential for implementing learning control in active vehicle suspensions, as well as for many other systems.

Modeling a Student's Behavior in a Tutorial-Like System Using Learning Automata

“…They have been used in game playing [4], [15], [16], pattern recognition [23], [47], object partitioning [36], [37], parameter optimization [7], [25], [45], [46], multiobjective analysis [22], telephony routing [24], [26], and priority assignments in a queuing system [20]. They have also been used in statistical decision making [21], [25], distribution approximation [1], natural language processing, modeling biological learning systems [57], string taxonomy [32], graph partitioning [33], distributed scheduling [51], network protocols (including conflict avoidance [39]) for LANs [40], photonic LANs [42], star networks [41], broadcast communication systems [43], dynamic channel allocation [43], tuning proportional-integral differential controllers [13], assigning capacities in prioritized networks [38], map learning [8], digital filter design [14], controlling client/server systems [44], adaptive signal processing [52], vehicle path control [58], the control of power systems [60], and vehicle suspension systems [10].…”

Generalized pursuit learning schemes: new families of continuous and discretized learning automata

“…They have been used in game playing [1]- [3], pattern recognition [10], [21], object partitioning [17], [18], parameter optimization [9], [28], [35], [54] and multi-objective analysis [43], telephony routing [11], [12], and priority assignments in a queuing system [7]. They have also been used in statistical decision making [9], [31], distribution approximation [40], natural language processing, modeling biological learning systems [26], string taxonomy [56], graph partitioning [57], distributed scheduling [39], network protocols (including conflict avoidance [44]) for LANs [36], photonic LANs [45], star networks [37], broadcast communication systems [38], dynamic channel allocation [38], tuning PID controllers [30], assigning capacities in prioritized networks [32], map learning [41], digital filter design [42], controlling client/server systems [46], adaptive signal processing [49], vehicle path control [50], and even the control of power systems [51] and vehicle suspension systems [52]. The beauty of incorporating LA in any particular application domain, is indeed, the elegance of the technology.…”

Continuous action reinforcement learning applied to vehicle suspension control

“…The stochastic Continuous Action Reinforcement Learning Automata (CARLA) algorithm described below was developed as an extension of the learning automata approach [7,8] and can be applied across a continuous range of actions. The algorithm is of a reward inaction type with multiple actions being implemented in a similar manner to interconnected learning automata, where the interconnection is through the dynamics of the 'environment', in this case the vehicle.…”

“…The specified ranges for these parameters have been based on the typical values reported in the literature on active and semi-active suspension control [7]. The action variable of each CARLA now corresponding to a controller parameter.…”