A self-learning strategy for artificial cognitive control systems

Beruvides, Gerardo; Juanes, Carmelo; Castaño, Fernando; Haber, Rodolfo E.

doi:10.1109/indin.2015.7281903

Cited by 21 publications

(12 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The Q-learning algorithm is a model-free reinforcement learning technique. Specifically, Q-learning can be used to find an optimal action-selection policy for any given (finite) Markov decision process [ 27 , 28 ]. It learns an action-value function that ultimately yields the expected utility of taking a given action in a given state and it then follows the optimal policy (off-policy learner).…”

Section: Self-tuning Methods For Reliability In Lidar Sensors Netwomentioning

confidence: 99%

Self-Tuning Method for Increased Obstacle Detection Reliability Based on Internet of Things LiDAR Sensor Models

Castaño

Beruvides

Villalonga

et al. 2018

Sensors

Self Cite

View full text Add to dashboard Cite

On-chip LiDAR sensors for vehicle collision avoidance are a rapidly expanding area of research and development. The assessment of reliable obstacle detection using data collected by LiDAR sensors has become a key issue that the scientific community is actively exploring. The design of a self-tuning methodology and its implementation are presented in this paper, to maximize the reliability of LiDAR sensors network for obstacle detection in the ‘Internet of Things’ (IoT) mobility scenarios. The Webots Automobile 3D simulation tool for emulating sensor interaction in complex driving environments is selected in order to achieve that objective. Furthermore, a model-based framework is defined that employs a point-cloud clustering technique, and an error-based prediction model library that is composed of a multilayer perceptron neural network, and k-nearest neighbors and linear regression models. Finally, a reinforcement learning technique, specifically a Q-learning method, is implemented to determine the number of LiDAR sensors that are required to increase sensor reliability for obstacle localization tasks. In addition, a IoT driving assistance user scenario, connecting a five LiDAR sensor network is designed and implemented to validate the accuracy of the computational intelligence-based framework. The results demonstrated that the self-tuning method is an appropriate strategy to increase the reliability of the sensor network while minimizing detection thresholds.

show abstract

Section: Self-tuning Methods For Reliability In Lidar Sensors Netwomentioning

confidence: 99%

Self-Tuning Method for Increased Obstacle Detection Reliability Based on Internet of Things LiDAR Sensor Models

Castaño

Beruvides

Villalonga

et al. 2018

Sensors

Self Cite

View full text Add to dashboard Cite

show abstract

“…The reward function is chosen for setting the best possible Q-value in 100 different scenes. Therefore, the function for updating the Q-values is [98]:…”

Section: Self-learning For Decision-making Evaluationmentioning

confidence: 99%

Sensor Reliability in Cyber-Physical Systems Using Internet-of-Things Data: A Review and Case Study

et al. 2019

Self Cite

View full text Add to dashboard Cite

Nowadays, reliability of sensors is one of the most important challenges for widespread application of Internet-of-things data in key emerging fields such as the automotive and manufacturing sectors. This paper presents a brief review of the main research and innovation actions at the European level, as well as some on-going research related to sensor reliability in cyber-physical systems (CPS). The research reported in this paper is also focused on the design of a procedure for evaluating the reliability of Internet-of-Things sensors in a cyber-physical system. The results of a case study of sensor reliability assessment in an autonomous driving scenario for the automotive sector are also shown. A co-simulation framework is designed in order to enable real-time interaction between virtual and real sensors. The case study consists of an IoT LiDAR-based collaborative map in order to assess the CPS-based co-simulation framework. Specifically, the sensor chosen is the Ibeo Lux 4-layer LiDAR sensor with IoT added capabilities. The modeling library for predicting error with machine learning methods is implemented at a local level, and a self-learning-procedure for decision-making based on Q-learning runs at a global level. The study supporting the experimental evaluation of the co-simulation framework is presented using simulated and real data. The results demonstrate the effectiveness of the proposed method for increasing sensor reliability in cyber-physical systems using Internet-of-Things data.

show abstract

“…The work is an excellent example of how the tools and concept of theoretical computer science and cognitive sciences can be applied to the control of complex industrial systems. Details about the design of the reinforced learning component of the cognitive controller are found in the Reference [65].…”

Section: Joint Applications Of Fc and Rlmentioning

confidence: 99%

Analysis and Adaptation of Q-Learning Algorithm to Expert Controls of a Solar Domestic Hot Water System

Bettoni

Soppelsa

Fedrizzi

et al. 2019

ASI

View full text Add to dashboard Cite

This paper discusses the development of a coupled Q-learning/fuzzy control algorithm to be applied to the control of solar domestic hot water systems. The controller brings the benefit of showing performance in line with the best reference controllers without the need for devoting time to modelling and simulations to tune its parameters before deployment. The performance of the proposed control algorithm was analysed in detail concerning the input membership function defining the fuzzy controller. The algorithm was compared to four standard reference control cases using three performance figures: the seasonal performance factor of the solar collectors, the seasonal performance factor of the system and the number of on/off cycles of the primary circulator. The work shows that the reinforced learning controller can find the best performing fuzzy controller within a family of controllers. It also shows how to increase the speed of the learning process by loading the controller with partial pre-existing information. The new controller performed significantly better than the best reference case with regard to the collectors’ performance factor (between 15% and 115%), and at the same time, to the number of on/off cycles of the primary circulator (1.2 per day down from 30 per day). Regarding the domestic hot water performance factor, the new controller performed about 11% worse than the best reference controller but greatly improved its on/off cycle figure (425 from 11,046). The decrease in performance was due to the choice of reward function, which was not selected for that purpose and it was blind to some of the factors influencing the system performance factor.

show abstract

A self-learning strategy for artificial cognitive control systems

Cited by 21 publications

References 18 publications

Self-Tuning Method for Increased Obstacle Detection Reliability Based on Internet of Things LiDAR Sensor Models

Self-Tuning Method for Increased Obstacle Detection Reliability Based on Internet of Things LiDAR Sensor Models

Sensor Reliability in Cyber-Physical Systems Using Internet-of-Things Data: A Review and Case Study

Analysis and Adaptation of Q-Learning Algorithm to Expert Controls of a Solar Domestic Hot Water System

Contact Info

Product

Resources

About