Reinforcement Learning Based Control Design for a Floating Piston Pneumatic Gearbox Actuator

Abstract

The project is also supported by the Hungarian Government and co-financed by the European Social Fund, EFOP-3.6.3-VEKOP-16-2017-00001: Talent management in autonomous vehicle control technologies ABSTRACT Electro-pneumatic actuators play an essential role in various areas of the industry, including heavy-duty vehicles. This article deals with the control problem of an Automatic Manual Transmission, where the actuator of the system is a double-acting floating-piston cylinder, with dedicated inner-position. During the control design of electro-pneumatic cylinders, one must implement a set-valued control on a nonlinear system, when, as in the present case, non-proportional valves provide the airflow. As both the system model itself and the qualitative control goals can be formulated as a Partially Observable Markov Decision Process, Machine learning frameworks are a conspicuous choice for handling such control problems. To this end, six different solutions are compared in the article, of which a new agent named PG-MCTS, using a modified version of the "Upper Confidence bound for Trees" algorithm, is also presented. The performance and strategic choice comparison of the six methods are carried out in a simulation environment. Validation tests performed on an actual transmission system and implemented on an automotive control unit to prove the applicability of the concept. In this case, a Policy Gradient agent, selected by implementation and computation capacity restrictions. The results show that the presented methods are suitable for the control of floating-piston cylinders and can be extended to other fluid mechanical actuators, or even different set-valued nonlinear control problems.

Engineering Applications of Artificial Intelligence in Mechanical Design and Optimization

“…In this instance, the policy gradient agent was chosen based on implementation limitations and computing capability. The findings indicate that the given approaches are appropriate for the control of floating piston cylinders and may be applied to other mechanical fluid drives and nonlinear control problems with fixed values [109].…”

Design of a Reinforcement Learning-Based Lane Keeping Planning Agent for Automated Vehicles

“…Moreover, consecutive steps do not represent strategically different scenarios in means like they do in chess or Go, which emphasize the introduced effect. This feature enables the direct use of PG's predicted values without the misleading effect, which initially would require the connection between the consecutive branches like in [42].…”

Considering PKI safety impact on network performance during V2X-based AD/ADAS function development processes

“…In order to prevent compromised messages from being transmitted during V2X communication, we must ensure that only parties with a valid digital certificate can join the communication. However, this communication process, secured by the PKI security framework, requires additional computation (time or resource), which we need to consider when developing the ADAS function [6] [7], especially for safety-critical applications that require real-time intervention (such as Autonomous Emergency Braking System -AEBS). Suppose we do not allocate additional computational resources.…”