Beyond temporal pooling : recurrence and temporal convolutions for gesture recognition in video
- Author
- Lionel Pigou (UGent) , Aäron van den Oord (UGent) , Sander Dieleman (UGent) , Mieke Van Herreweghe (UGent) and Joni Dambre (UGent)
- Organization
- Abstract
- Recent studies have demonstrated the power of recurrent neural networks for machine translation, image captioning and speech recognition. For the task of capturing temporal structure in video, however, there still remain numerous open research questions. Current research suggests using a simple temporal feature pooling strategy to take into account the temporal aspect of video. We demonstrate that this method is not sufficient for gesture recognition, where temporal information is more discriminative compared to general video classification tasks. We explore deep architectures for gesture recognition in video and propose a new end-to-end trainable neural network architecture incorporating temporal convolutions and bidirectional recurrence. Our main contributions are twofold; first, we show that recurrence is crucial for this task; second, we show that adding temporal convolutions leads to significant improvements. We evaluate the different approaches on the Montalbano gesture recognition dataset, where we achieve state-of-the-art results.
- Keywords
- Gesture recognition, Deep neural networks
Downloads
-
(...).pdf
- full text
- |
- UGent only
- |
- |
- 1.31 MB
Citation
Please use this url to cite or link to this publication: http://hdl.handle.net/1854/LU-8516623
- MLA
- Pigou, Lionel, et al. “Beyond Temporal Pooling : Recurrence and Temporal Convolutions for Gesture Recognition in Video.” INTERNATIONAL JOURNAL OF COMPUTER VISION, vol. 126, no. 2–4, Springer US, 2018, pp. 430–39, doi:10.1007/s11263-016-0957-7.
- APA
- Pigou, L., van den Oord, A., Dieleman, S., Van Herreweghe, M., & Dambre, J. (2018). Beyond temporal pooling : recurrence and temporal convolutions for gesture recognition in video. INTERNATIONAL JOURNAL OF COMPUTER VISION, 126(2–4), 430–439. https://doi.org/10.1007/s11263-016-0957-7
- Chicago author-date
- Pigou, Lionel, Aäron van den Oord, Sander Dieleman, Mieke Van Herreweghe, and Joni Dambre. 2018. “Beyond Temporal Pooling : Recurrence and Temporal Convolutions for Gesture Recognition in Video.” INTERNATIONAL JOURNAL OF COMPUTER VISION 126 (2–4): 430–39. https://doi.org/10.1007/s11263-016-0957-7.
- Chicago author-date (all authors)
- Pigou, Lionel, Aäron van den Oord, Sander Dieleman, Mieke Van Herreweghe, and Joni Dambre. 2018. “Beyond Temporal Pooling : Recurrence and Temporal Convolutions for Gesture Recognition in Video.” INTERNATIONAL JOURNAL OF COMPUTER VISION 126 (2–4): 430–439. doi:10.1007/s11263-016-0957-7.
- Vancouver
- 1.Pigou L, van den Oord A, Dieleman S, Van Herreweghe M, Dambre J. Beyond temporal pooling : recurrence and temporal convolutions for gesture recognition in video. INTERNATIONAL JOURNAL OF COMPUTER VISION. 2018;126(2–4):430–9.
- IEEE
- [1]L. Pigou, A. van den Oord, S. Dieleman, M. Van Herreweghe, and J. Dambre, “Beyond temporal pooling : recurrence and temporal convolutions for gesture recognition in video,” INTERNATIONAL JOURNAL OF COMPUTER VISION, vol. 126, no. 2–4, pp. 430–439, 2018.
@article{8516623,
abstract = {{Recent studies have demonstrated the power of recurrent neural networks for machine translation, image captioning and speech recognition. For the task of capturing temporal structure in video, however, there still remain numerous open research questions. Current research suggests using a simple temporal feature pooling strategy to take into account the temporal aspect of video. We demonstrate that this method is not sufficient for gesture recognition, where temporal information is more discriminative compared to general video classification tasks. We explore deep architectures for gesture recognition in video and propose a new end-to-end trainable neural network architecture incorporating temporal convolutions and bidirectional recurrence. Our main contributions are twofold; first, we show that recurrence is crucial for this task; second, we show that adding temporal convolutions leads to significant improvements. We evaluate the different approaches on the Montalbano gesture recognition dataset, where we achieve state-of-the-art results.}},
author = {{Pigou, Lionel and van den Oord, Aäron and Dieleman, Sander and Van Herreweghe, Mieke and Dambre, Joni}},
issn = {{0920-5691}},
journal = {{INTERNATIONAL JOURNAL OF COMPUTER VISION}},
keywords = {{Gesture recognition,Deep neural networks}},
language = {{eng}},
number = {{2-4}},
pages = {{430--439}},
publisher = {{Springer US}},
title = {{Beyond temporal pooling : recurrence and temporal convolutions for gesture recognition in video}},
url = {{http://doi.org/10.1007/s11263-016-0957-7}},
volume = {{126}},
year = {{2018}},
}
- Altmetric
- View in Altmetric
- Web of Science
- Times cited: