Optimizing cascaded control of mechatronic systems through constrained residual reinforcement learning
- Author
- Tom Staessens (UGent) , Tom Lefebvre (UGent) and Guillaume Crevecoeur (UGent)
- Organization
- Project
- Abstract
- Cascaded control structures are prevalent in industrial systems with many disturbances to obtain stable control but are cumbersome and challenging to tune. In this work, we propose cascaded constrained residual reinforcement learning (RL), an intuitive method that allows to improve the performance of a cascaded control structure while maintaining safe operation at all times. We draw inspiration from the constrained residual RL framework, in which a constrained reinforcement learning agent learns corrective adaptations to a base controller’s output to increase optimality. We first revisit the interplay between the residual agent and the baseline controller and subsequently extend this to the cascaded case. We analyze the differences and challenges this structure brings and derive some principle insights from this into the stability and operation of the cascaded residual architecture. Next, we propose a novel actor structure to enable efficient learning under the cascaded setting. We show that the standard algorithm is suboptimal for application to cascaded control structures and validate our method on a high-fidelity simulator of a dual motor drivetrain, resulting in a performance improvement of 14.7% on average, with only a minor decrease in performance occurring during the training phase. We study the different principles constituting the method and examine and validate their contribution to the algorithm’s performance under the considered cascaded control structure.
- Keywords
- Electrical and Electronic Engineering, Industrial and Manufacturing Engineering, Control and Optimization, Mechanical Engineering, Computer Science (miscellaneous), Control and Systems Engineering, mechatronics, motion control, cascaded control, reinforcement learning, (RL), uncertain systems, STRATEGY, DESIGN, GAME
Downloads
-
Publisher version.pdf
- full text (Published version)
- |
- open access
- |
- |
- 741.34 KB
Citation
Please use this url to cite or link to this publication: http://hdl.handle.net/1854/LU-01GX6M3GSZ40NPSH8AGNWYAJYK
- MLA
- Staessens, Tom, et al. “Optimizing Cascaded Control of Mechatronic Systems through Constrained Residual Reinforcement Learning.” MACHINES, vol. 11, no. 3, 2023, doi:10.3390/machines11030402.
- APA
- Staessens, T., Lefebvre, T., & Crevecoeur, G. (2023). Optimizing cascaded control of mechatronic systems through constrained residual reinforcement learning. MACHINES, 11(3). https://doi.org/10.3390/machines11030402
- Chicago author-date
- Staessens, Tom, Tom Lefebvre, and Guillaume Crevecoeur. 2023. “Optimizing Cascaded Control of Mechatronic Systems through Constrained Residual Reinforcement Learning.” MACHINES 11 (3). https://doi.org/10.3390/machines11030402.
- Chicago author-date (all authors)
- Staessens, Tom, Tom Lefebvre, and Guillaume Crevecoeur. 2023. “Optimizing Cascaded Control of Mechatronic Systems through Constrained Residual Reinforcement Learning.” MACHINES 11 (3). doi:10.3390/machines11030402.
- Vancouver
- 1.Staessens T, Lefebvre T, Crevecoeur G. Optimizing cascaded control of mechatronic systems through constrained residual reinforcement learning. MACHINES. 2023;11(3).
- IEEE
- [1]T. Staessens, T. Lefebvre, and G. Crevecoeur, “Optimizing cascaded control of mechatronic systems through constrained residual reinforcement learning,” MACHINES, vol. 11, no. 3, 2023.
@article{01GX6M3GSZ40NPSH8AGNWYAJYK,
abstract = {{Cascaded control structures are prevalent in industrial systems with many disturbances to obtain stable control but are cumbersome and challenging to tune. In this work, we propose cascaded constrained residual reinforcement learning (RL), an intuitive method that allows to improve the performance of a cascaded control structure while maintaining safe operation at all times. We draw inspiration from the constrained residual RL framework, in which a constrained reinforcement learning agent learns corrective adaptations to a base controller’s output to increase optimality. We first revisit the interplay between the residual agent and the baseline controller and subsequently extend this to the cascaded case. We analyze the differences and challenges this structure brings and derive some principle insights from this into the stability and operation of the cascaded residual architecture. Next, we propose a novel actor structure to enable efficient learning under the cascaded setting. We show that the standard algorithm is suboptimal for application to cascaded control structures and validate our method on a high-fidelity simulator of a dual motor drivetrain, resulting in a performance improvement of 14.7% on average, with only a minor decrease in performance occurring during the training phase. We study the different principles constituting the method and examine and validate their contribution to the algorithm’s performance under the considered cascaded control structure.}},
articleno = {{402}},
author = {{Staessens, Tom and Lefebvre, Tom and Crevecoeur, Guillaume}},
issn = {{2075-1702}},
journal = {{MACHINES}},
keywords = {{Electrical and Electronic Engineering,Industrial and Manufacturing Engineering,Control and Optimization,Mechanical Engineering,Computer Science (miscellaneous),Control and Systems Engineering,mechatronics,motion control,cascaded control,reinforcement learning,(RL),uncertain systems,STRATEGY,DESIGN,GAME}},
language = {{eng}},
number = {{3}},
pages = {{19}},
title = {{Optimizing cascaded control of mechatronic systems through constrained residual reinforcement learning}},
url = {{http://doi.org/10.3390/machines11030402}},
volume = {{11}},
year = {{2023}},
}
- Altmetric
- View in Altmetric
- Web of Science
- Times cited: