Fixed structure controllers (such as proportional-integral-derivative
controllers) are used extensively in industry. Finding a practical and
versatile method to tune these controllers, particularly with imprecise
process models and limited online computational resources, is an
industrially relevant problem which could improve the efficiency of many
plants. In this paper, we present two flexible neural network-based
approaches capable of tuning any fixed structure controller for any
control objective and process model and compare their advantages and
disadvantages. The first approach is derived from supervised learning
and classical optimization techniques, while the second approach applies
techniques used in deep reinforcement learning. Both approaches
incorporate model uncertainties when selecting controller parameters,
reducing the need for costly experiments to precisely estimate model
parameters in a plant. Both methods are also computationally efficient
online, enabling their widespread usage.
Meta-learning is a branch of machine learning which trains neural network models to synthesize a wide variety of data in order to rapidly solve new problems. In process control, many systems have similar and well-understood dynamics, which suggests it is feasible to create a generalizable controller through meta-learning. In this work, we formulate a meta reinforcement learning (meta-RL) control strategy that takes advantage of known, offline information for training, such as the system gain or time constant, yet efficiently controls novel systems in a completely model-free fashion. Our meta-RL agent has a recurrent structure that accumulates "context" for its current dynamics through a hidden state variable. This end-to-end architecture enables the agent to automatically adapt to changes in the process dynamics. Moreover, the same agent can be deployed on systems with previously unseen nonlinearities and timescales. In tests reported here, the meta-RL agent was trained entirely offline, yet produced excellent results in novel settings. A key design element is the ability to leverage model-based information offline during training, while maintaining a model-free policy structure for interacting with novel environments. To illustrate the approach, we take the actions proposed by the meta-RL agent to be changes to gains of a proportional-integral controller, resulting in a generalized, adaptive, closed-loop tuning strategy. Meta-learning is a promising approach for constructing sample-efficient intelligent controllers.
Meta-learning is a branch of machine learning which aims to synthesize data from a distribution of related tasks to efficiently solve new ones. In process control, many systems have similar and well-understood dynamics, which suggests it is feasible to create a generalizable controller through meta-learning. In this work, we formulate a meta reinforcement learning (meta-RL) control strategy that takes advantage of known, offline information for training, such as a model structure. The meta-RL agent is trained over a distribution of model parameters, rather than a single model, enabling the agent to automatically adapt to changes in the process dynamics while maintaining performance. A key design element is the ability to leverage model-based information offline during training, while maintaining a model-free policy structure for interacting with new environments. Our previous work has demonstrated how this approach can be applied to the industrially-relevant problem of tuning proportional-integral controllers to control first order processes. In this work, we briefly reintroduce our methodology and demonstrate how it can be extended to proportional-integral-derivative controllers and second order systems.
The COVID-19 pandemic has shifted a significant number of on-campus and in-person activitiesto an online, virtual setting. This has caused difficulty in achieving the same learning outcomes in the absence of in-person interaction, particularly for lab courses. In 2014, the Chemical and Biological Engineering department at The University of British Columbia developed and implemented a Teaching Laboratory Data Management (TLDM) system to improve the delivery andeffectiveness of lab-based courses. The TLDM system guides students through experimental calculations and automates a significant amount of grading for instructors. The TLDM system aided the department to better adapt to online lab courses and was integral to the virtual instruction of lab courses during the COVID-19 pandemic. Through conducting surveys and group feedback sessions with students, teaching assistants (TAs) and instructors, we found that the TLDM systemhas been well received by all three groups of stakeholders for use in online lab courses, and provides several key benefits. Namely, it helps students understand the calculations involved in experiments and provides an effective substitute for in-person lab activities, while reducing workload for TAs and instructors. The TLDM system can potentially be a great tool to complementonline courses around the world in an inexpensive way.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.