Learning temporal task specifications from demonstrations
- Author
- Mattijs Baert (UGent) , Sam Leroux (UGent) and Pieter Simoens (UGent)
- Organization
- Project
- Abstract
- As we progress towards real-world deployment, the critical need for interpretability in reinforcement learning algorithms grows more pivotal, ensuring the safety and reliability of intelligent agents. This paper tackles the challenge of acquiring task specifications in linear temporal logic through expert demonstrations, aiming to alleviate the burdensome task of specification engineering. The rich semantics of temporal logics serve as an interpretable framework for delineating intricate, multi-stage tasks. We propose a method which iteratively learns a task specification and a nominal policy solving this task. In each iteration, the task specification is refined to better distinguish expert trajectories from trajectories sampled from the nominal policy. With this process we obtain a concise and interpretable task specification. Unlike previous work, our method is capable of learning directly from trajectories in the original state space and does not require predefined atomic propositions. We showcase the effectiveness of our method on multiple tasks in both an office and a Minecraft-inspired environment.
- Keywords
- Learning From Demonstrations, Linear Temporal Logic, Reinforcement Learning, Inverse Reinforcement Learning, LOGIC
Downloads
-
(...).pdf
- full text (Published version)
- |
- UGent only
- |
- |
- 825.76 KB
Citation
Please use this url to cite or link to this publication: http://hdl.handle.net/1854/LU-01J914A7MG4B2042HT11123XAH
- MLA
- Baert, Mattijs, et al. “Learning Temporal Task Specifications from Demonstrations.” EXPLAINABLE AND TRANSPARENT AI AND MULTI-AGENT SYSTEMS, EXTRAAMAS 2024, vol. 14847, Springer, 2024, pp. 81–98, doi:10.1007/978-3-031-70074-3_5.
- APA
- Baert, M., Leroux, S., & Simoens, P. (2024). Learning temporal task specifications from demonstrations. EXPLAINABLE AND TRANSPARENT AI AND MULTI-AGENT SYSTEMS, EXTRAAMAS 2024, 14847, 81–98. https://doi.org/10.1007/978-3-031-70074-3_5
- Chicago author-date
- Baert, Mattijs, Sam Leroux, and Pieter Simoens. 2024. “Learning Temporal Task Specifications from Demonstrations.” In EXPLAINABLE AND TRANSPARENT AI AND MULTI-AGENT SYSTEMS, EXTRAAMAS 2024, 14847:81–98. Cham: Springer. https://doi.org/10.1007/978-3-031-70074-3_5.
- Chicago author-date (all authors)
- Baert, Mattijs, Sam Leroux, and Pieter Simoens. 2024. “Learning Temporal Task Specifications from Demonstrations.” In EXPLAINABLE AND TRANSPARENT AI AND MULTI-AGENT SYSTEMS, EXTRAAMAS 2024, 14847:81–98. Cham: Springer. doi:10.1007/978-3-031-70074-3_5.
- Vancouver
- 1.Baert M, Leroux S, Simoens P. Learning temporal task specifications from demonstrations. In: EXPLAINABLE AND TRANSPARENT AI AND MULTI-AGENT SYSTEMS, EXTRAAMAS 2024. Cham: Springer; 2024. p. 81–98.
- IEEE
- [1]M. Baert, S. Leroux, and P. Simoens, “Learning temporal task specifications from demonstrations,” in EXPLAINABLE AND TRANSPARENT AI AND MULTI-AGENT SYSTEMS, EXTRAAMAS 2024, Auckland, New Zealand, 2024, vol. 14847, pp. 81–98.
@inproceedings{01J914A7MG4B2042HT11123XAH,
abstract = {{As we progress towards real-world deployment, the critical need for interpretability in reinforcement learning algorithms grows more pivotal, ensuring the safety and reliability of intelligent agents. This paper tackles the challenge of acquiring task specifications in linear temporal logic through expert demonstrations, aiming to alleviate the burdensome task of specification engineering. The rich semantics of temporal logics serve as an interpretable framework for delineating intricate, multi-stage tasks. We propose a method which iteratively learns a task specification and a nominal policy solving this task. In each iteration, the task specification is refined to better distinguish expert trajectories from trajectories sampled from the nominal policy. With this process we obtain a concise and interpretable task specification. Unlike previous work, our method is capable of learning directly from trajectories in the original state space and does not require predefined atomic propositions. We showcase the effectiveness of our method on multiple tasks in both an office and a Minecraft-inspired environment.}},
author = {{Baert, Mattijs and Leroux, Sam and Simoens, Pieter}},
booktitle = {{EXPLAINABLE AND TRANSPARENT AI AND MULTI-AGENT SYSTEMS, EXTRAAMAS 2024}},
isbn = {{9783031700736}},
issn = {{0302-9743}},
keywords = {{Learning From Demonstrations,Linear Temporal Logic,Reinforcement Learning,Inverse Reinforcement Learning,LOGIC}},
language = {{eng}},
location = {{Auckland, New Zealand}},
pages = {{81--98}},
publisher = {{Springer}},
title = {{Learning temporal task specifications from demonstrations}},
url = {{http://doi.org/10.1007/978-3-031-70074-3_5}},
volume = {{14847}},
year = {{2024}},
}
- Altmetric
- View in Altmetric
- Web of Science
- Times cited: