Preventing catastrophic forgetting using prior transfer in physics informed Bayesian neural networks
- Author
- Cedric Van Heck (UGent) , Annelies Coene (UGent) and Guillaume Crevecoeur (UGent)
- Organization
- Abstract
- Predictive models can be integrated in the sensing and monitoring methodologies of mechatronic systems in operation. When systems change or are subject to varying operating conditions, adaptivity of the models is needed. The goal of this paper is to enable this adaptivity by presenting a framework for continual learning. The framework aims to transfer and remember information from previously learned systems when a model is updated to new operating conditions. We achieve this by means of the following three key mechanisms. We first include physical information about the system, heavily regularizing the model output. Secondly, the usage of epistemic uncertainty, used as an indicator of the changing system, shows to what extend a transfer is desired. Last but not least the usage of a prior within a Bayesian framework allows to regularize models further according to previously obtained information. The last two principles are enabled thanks to the use of Bayesian neural networks. The methodology will be applied to a cam-follower system in a simulation environment, where results show that previously trained systems are better remembered with an increase of 72% compared to normal training procedures.
- Keywords
- Bayesian, neural networks, catastrophic forgetting, continual learning, hybrid modelling, physics inspired, mechatronic system, FRICTION COMPENSATION, DRIVEN, DESIGN, MPC
Downloads
-
Preventing Catastrophic Forgetting using Prior Transfer in Physics Informed Bayesian Neural Networks.pdf
- full text (Accepted manuscript)
- |
- open access
- |
- |
- 2.02 MB
-
(...).pdf
- full text (Published version)
- |
- UGent only
- |
- |
- 2.78 MB
Citation
Please use this url to cite or link to this publication: http://hdl.handle.net/1854/LU-8764739
- MLA
- Van Heck, Cedric, et al. “Preventing Catastrophic Forgetting Using Prior Transfer in Physics Informed Bayesian Neural Networks.” 2022 IEEE/ASME International Conference on Advanced Intelligent Mechatronics (AIM), IEEE, 2022, pp. 650–57, doi:10.1109/aim52237.2022.9863403.
- APA
- Van Heck, C., Coene, A., & Crevecoeur, G. (2022). Preventing catastrophic forgetting using prior transfer in physics informed Bayesian neural networks. 2022 IEEE/ASME International Conference on Advanced Intelligent Mechatronics (AIM), 650–657. https://doi.org/10.1109/aim52237.2022.9863403
- Chicago author-date
- Van Heck, Cedric, Annelies Coene, and Guillaume Crevecoeur. 2022. “Preventing Catastrophic Forgetting Using Prior Transfer in Physics Informed Bayesian Neural Networks.” In 2022 IEEE/ASME International Conference on Advanced Intelligent Mechatronics (AIM), 650–57. IEEE. https://doi.org/10.1109/aim52237.2022.9863403.
- Chicago author-date (all authors)
- Van Heck, Cedric, Annelies Coene, and Guillaume Crevecoeur. 2022. “Preventing Catastrophic Forgetting Using Prior Transfer in Physics Informed Bayesian Neural Networks.” In 2022 IEEE/ASME International Conference on Advanced Intelligent Mechatronics (AIM), 650–657. IEEE. doi:10.1109/aim52237.2022.9863403.
- Vancouver
- 1.Van Heck C, Coene A, Crevecoeur G. Preventing catastrophic forgetting using prior transfer in physics informed Bayesian neural networks. In: 2022 IEEE/ASME International Conference on Advanced Intelligent Mechatronics (AIM). IEEE; 2022. p. 650–7.
- IEEE
- [1]C. Van Heck, A. Coene, and G. Crevecoeur, “Preventing catastrophic forgetting using prior transfer in physics informed Bayesian neural networks,” in 2022 IEEE/ASME International Conference on Advanced Intelligent Mechatronics (AIM), Sapporo, Japan, 2022, pp. 650–657.
@inproceedings{8764739,
abstract = {{Predictive models can be integrated in the sensing and monitoring methodologies of mechatronic systems in operation. When systems change or are subject to varying operating conditions, adaptivity of the models is needed. The goal of this paper is to enable this adaptivity by presenting a framework for continual learning. The framework aims to transfer and remember information from previously learned systems when a model is updated to new operating conditions. We achieve this by means of the following three key mechanisms. We first include physical information about the system, heavily regularizing the model output. Secondly, the usage of epistemic uncertainty, used as an indicator of the changing system, shows to what extend a transfer is desired. Last but not least the usage of a prior within a Bayesian framework allows to regularize models further according to previously obtained information. The last two principles are enabled thanks to the use of Bayesian neural networks. The methodology will be applied to a cam-follower system in a simulation environment, where results show that previously trained systems are better remembered with an increase of 72% compared to normal training procedures.}},
author = {{Van Heck, Cedric and Coene, Annelies and Crevecoeur, Guillaume}},
booktitle = {{2022 IEEE/ASME International Conference on Advanced Intelligent Mechatronics (AIM)}},
isbn = {{9781665413084}},
issn = {{2159-6247}},
keywords = {{Bayesian,neural networks,catastrophic forgetting,continual learning,hybrid modelling,physics inspired,mechatronic system,FRICTION COMPENSATION,DRIVEN,DESIGN,MPC}},
language = {{eng}},
location = {{Sapporo, Japan}},
pages = {{650--657}},
publisher = {{IEEE}},
title = {{Preventing catastrophic forgetting using prior transfer in physics informed Bayesian neural networks}},
url = {{http://dx.doi.org/10.1109/aim52237.2022.9863403}},
year = {{2022}},
}
- Altmetric
- View in Altmetric
- Web of Science
- Times cited: