Reinforcement learning for an enhanced energy flexibility controller incorporating predictive safety filter and adaptive policy updates
- Author
- Siebe Paesschesoone (UGent) , Nezmin Kayedpour (UGent) , Carlo Manna and Guillaume Crevecoeur (UGent)
- Organization
- Abstract
- This paper presents a novel data-driven approach that leverages reinforcement learning to enhance the efficiency and safety of existing energy flexibility controllers, addressing challenges posed by the dynamic and uncertain nature of modern energy landscapes. With the increasing integration of renewable energy sources, conventional controllers struggle to maintain both safety and optimality. Our proposed approach introduces two significant contributions to standard RL approaches: a data-driven predictive safety filter and an online changepoint detection and policy updating module. Through continuous constraint satisfaction, the predictive safety filter guarantees absolute safety of the proposed controller. Meanwhile, the changepoint detection and policy updating module, inspired by the concept of continual learning, enhances the controller's adaptivity to non-stationary environments. By identifying changes in the environment, it triggers relearning of the agent, making the controller resilient to evolving conditions. Validation of our approach is conducted on a gridconnected PV-battery-load system, demonstrating its effectiveness in simultaneously improving safety and performance over traditional learning methods. More specifically, the proposed solution was able to increase the energy flexibility by reducing energy costs with 9.3%.
- Keywords
- Energy flexibility control, Safe continual reinforcement learning, Predictive safety filter, Changepoint detection, Policy updating, COMBINATION, LOCATION
Downloads
-
published.pdf
- full text (Published version)
- |
- open access
- |
- |
- 2.38 MB
Citation
Please use this url to cite or link to this publication: http://hdl.handle.net/1854/LU-01HYDW95C1WMZ5YNVYJWB7P8EB
- MLA
- Paesschesoone, Siebe, et al. “Reinforcement Learning for an Enhanced Energy Flexibility Controller Incorporating Predictive Safety Filter and Adaptive Policy Updates.” APPLIED ENERGY, vol. 368, 2024, doi:10.1016/j.apenergy.2024.123507.
- APA
- Paesschesoone, S., Kayedpour, N., Manna, C., & Crevecoeur, G. (2024). Reinforcement learning for an enhanced energy flexibility controller incorporating predictive safety filter and adaptive policy updates. APPLIED ENERGY, 368. https://doi.org/10.1016/j.apenergy.2024.123507
- Chicago author-date
- Paesschesoone, Siebe, Nezmin Kayedpour, Carlo Manna, and Guillaume Crevecoeur. 2024. “Reinforcement Learning for an Enhanced Energy Flexibility Controller Incorporating Predictive Safety Filter and Adaptive Policy Updates.” APPLIED ENERGY 368. https://doi.org/10.1016/j.apenergy.2024.123507.
- Chicago author-date (all authors)
- Paesschesoone, Siebe, Nezmin Kayedpour, Carlo Manna, and Guillaume Crevecoeur. 2024. “Reinforcement Learning for an Enhanced Energy Flexibility Controller Incorporating Predictive Safety Filter and Adaptive Policy Updates.” APPLIED ENERGY 368. doi:10.1016/j.apenergy.2024.123507.
- Vancouver
- 1.Paesschesoone S, Kayedpour N, Manna C, Crevecoeur G. Reinforcement learning for an enhanced energy flexibility controller incorporating predictive safety filter and adaptive policy updates. APPLIED ENERGY. 2024;368.
- IEEE
- [1]S. Paesschesoone, N. Kayedpour, C. Manna, and G. Crevecoeur, “Reinforcement learning for an enhanced energy flexibility controller incorporating predictive safety filter and adaptive policy updates,” APPLIED ENERGY, vol. 368, 2024.
@article{01HYDW95C1WMZ5YNVYJWB7P8EB,
abstract = {{This paper presents a novel data-driven approach that leverages reinforcement learning to enhance the efficiency and safety of existing energy flexibility controllers, addressing challenges posed by the dynamic and uncertain nature of modern energy landscapes. With the increasing integration of renewable energy sources, conventional controllers struggle to maintain both safety and optimality. Our proposed approach introduces two significant contributions to standard RL approaches: a data-driven predictive safety filter and an online changepoint detection and policy updating module. Through continuous constraint satisfaction, the predictive safety filter guarantees absolute safety of the proposed controller. Meanwhile, the changepoint detection and policy updating module, inspired by the concept of continual learning, enhances the controller's adaptivity to non-stationary environments. By identifying changes in the environment, it triggers relearning of the agent, making the controller resilient to evolving conditions. Validation of our approach is conducted on a gridconnected PV-battery-load system, demonstrating its effectiveness in simultaneously improving safety and performance over traditional learning methods. More specifically, the proposed solution was able to increase the energy flexibility by reducing energy costs with 9.3%.}},
articleno = {{123507}},
author = {{Paesschesoone, Siebe and Kayedpour, Nezmin and Manna, Carlo and Crevecoeur, Guillaume}},
issn = {{0306-2619}},
journal = {{APPLIED ENERGY}},
keywords = {{Energy flexibility control,Safe continual reinforcement learning,Predictive safety filter,Changepoint detection,Policy updating,COMBINATION,LOCATION}},
language = {{eng}},
pages = {{9}},
title = {{Reinforcement learning for an enhanced energy flexibility controller incorporating predictive safety filter and adaptive policy updates}},
url = {{http://doi.org/10.1016/j.apenergy.2024.123507}},
volume = {{368}},
year = {{2024}},
}
- Altmetric
- View in Altmetric
- Web of Science
- Times cited: