A neural network controller using reinforcement learning method for ATM traffic policing

Abstract

In this paper, a Neural Network (NN) controller using reinforcement learning method for ATM traffic policing is presented. The mechanism is based upon an accurate estimation of the probability density function (pdf) of the traffic via its count process and the design of a NN critic function capable of evaluating the system performance in terms of the pdf violation. The pdf-based policing is made possible only by NNs. This is due to the fact that pdf policing requires complex calculations, in real-time, at very high rates which is not feasible via conventional mathematical approaches. The architecture of the NN policing mechanism is composed of critic function and control function. The critic function uses two inter-connected NNs. The first NN is trained to learn the pdf of an "ideal nonviolating" traffic, whereas the seeond NN is trained to capture the "actual" characteristics of the "actual" offered traffic during the progress of the call. The output of both NNs (which is an accurate estimate of the traffic bit-rate fluctuations in the next measurement period) is compared. Consequently, an error signal is generated whenever the pdf of the offered traffic violates its "ideal" case. The error signal is, then, used by the critic function to produce an evaluation signal based upon a certain performance measure. A reinforcement learning method uses the evaluation signal to adjust the weights of a third NN, in the control function that generates a control signal capable of maximizing the system performance (i.e., minimization of the traffic violations). The reported results prove that our policing mechanism is very effective in detecting (and controlling) all possible kinds of traffic violations. I, INTRODUCTIONAsynchronous Transfer Mode (ATM) is the technology recommended by CCITT [I] to provide voice, data, image, and video services using a single integrated broadband network. The ATM solution is flexible enough to support a diverse mixture of multimedia traffic with different correlations and burstiness properties. Not only does the different types of multimedia traffic differ in their statistical characteristics but also in their service requirements, from the network, specified by the quality of service (QOS). Due to the absence of a channel structure in ATM-based networks, the user's traffic can easily overwhelm the network resources leading to serious congestion problems. Traffic enforcement (policing) is required to control the user's traffic to an agreed "contract" determined at the call set-up phase. The policing mechanism controls the traffic via a set of user declared descriptors or parameters such as the peak bit-rate, mean bit-rate, and the burst duration. These parameters are also used by the connection acceptance control (CAC) algorithm that 0-7803-1828-5/94 $4.00 0 1994 IEEE decides to accept a new connection on the link if the required QOS is guaranteed for the new connection while maintaining the QOS of the existing connections. Hence, at the call set-up phase, a traffic contract is "agreed-upon" be...

Genetic algorithm-based neural fuzzy decision tree for mixed scheduling in ATM networks

“…Hence, by using the chain rule, we derive the error transmission in Layer 3 and the update of the parameters and in Layer 2 in the following. Layer 3: Only the error signal needs to be computed in this layer (21) where (22) if if .…”

“…Because the condition for the schedulability test of the mixed scheduling involves a large set of inequalities and no analytic closed-form solution can be obtained, we propose a genetic algorithm-based neural fuzzy decision tree (GANFDT) to realize the mixed scheduling scheme efficiently. Neural networks and fuzzy systems have been applied for ATM traffic control recently [2], [5], [22], because they are thought to have a lot potential in this area, especially for their learning and adaptive capabilities. The neural fuzzy networks require no explicit model of the traffic, and the parallel structure of neural networks can be exploited in hardware implementations, which provides short response time.…”

Neural networks in teletraffic control: pro et contra?

“…A policing mechanism using NNs. called neural network traffic enforcement mechanism, is proposed in [71]. it is based on the estimation of the probability density function of the multimedia traffic via its count process.…”

“…The controller uses the number of cells in the multiplexer buffer as a measure of potential congestion problems. Reinforcement learning, similar to that in [71], is applied to generate a control signal that is fed back to the input sources in order to adapt their rates. Another approach to congestion control using the NN is presented in [73].…”