Advanced search
1 file | 547.86 KB Add to list

Behaviourally cloning river raid agents

Author
Organization
Abstract
We investigate the feasibility and difficulties of using behavioural cloning to obtain player models using the 1982 video game River Raid. We attempt to clone both virtual game-playing agents (a fixed (non-improving) reinforcement learning agent and a random agent sampling actions uniformly) as well as an actual human agent. The behavioural clones' performance is evaluated on the micro-level through comparison of the state-conditioned and unconditional action distributions, and on the macro-level by comparing the (cloned) agents' survival time and score per episode. Using our methodology, cloning virtual agents seems feasible to varying extents, even with somewhat limited amounts of data. However, our method fails to create reliable behavioural clones of human players. We conclude with a discussion of some of the more important reasons that might cause this: a lack of training data, the problem of covariate shift, and improving and inconsistent play-style over time.
Keywords
Behavioural cloning, Imitation learning, River Raid, Reinforcement learning

Downloads

  • Behaviourally Cloning River Raid Agents.pdf
    • full text (Published version)
    • |
    • open access
    • |
    • PDF
    • |
    • 547.86 KB

Citation

Please use this url to cite or link to this publication:

MLA
Diels, Laurens, and Hussain Kazmi. “Behaviourally Cloning River Raid Agents.” 35th AAAI Conference on Artificial Intelligence, Proceedings, 2021.
APA
Diels, L., & Kazmi, H. (2021). Behaviourally cloning river raid agents. In 35th AAAI Conference on Artificial Intelligence, Proceedings. Virtual online conference.
Chicago author-date
Diels, Laurens, and Hussain Kazmi. 2021. “Behaviourally Cloning River Raid Agents.” In 35th AAAI Conference on Artificial Intelligence, Proceedings.
Chicago author-date (all authors)
Diels, Laurens, and Hussain Kazmi. 2021. “Behaviourally Cloning River Raid Agents.” In 35th AAAI Conference on Artificial Intelligence, Proceedings.
Vancouver
1.
Diels L, Kazmi H. Behaviourally cloning river raid agents. In: 35th AAAI Conference on Artificial Intelligence, Proceedings. 2021.
IEEE
[1]
L. Diels and H. Kazmi, “Behaviourally cloning river raid agents,” in 35th AAAI Conference on Artificial Intelligence, Proceedings, Virtual online conference, 2021.
@inproceedings{8691939,
  abstract     = {{We investigate the feasibility and difficulties of using behavioural cloning to obtain player models using the 1982 video game River Raid. We attempt to clone both virtual game-playing agents (a fixed (non-improving) reinforcement learning agent and a random agent sampling actions uniformly) as well as an actual human agent. The behavioural clones' performance is evaluated on the micro-level through comparison of the state-conditioned and unconditional action distributions, and on the macro-level by comparing the (cloned) agents' survival time and score per episode. Using our methodology, cloning virtual agents seems feasible to varying extents, even with somewhat limited amounts of data. However, our method fails to create reliable behavioural clones of human players. We conclude with a discussion of some of the more important reasons that might cause this: a lack of training data, the problem of covariate shift, and improving and inconsistent play-style over time.}},
  author       = {{Diels, Laurens and Kazmi, Hussain}},
  booktitle    = {{35th AAAI Conference on Artificial Intelligence, Proceedings}},
  keywords     = {{Behavioural cloning,Imitation learning,River Raid,Reinforcement learning}},
  language     = {{eng}},
  location     = {{Virtual online conference}},
  pages        = {{7}},
  title        = {{Behaviourally cloning river raid agents}},
  url          = {{http://aaai-rlg.mlanctot.info/papers/AAAI21-RLG_paper_3.pdf}},
  year         = {{2021}},
}