Ghent University Academic Bibliography

Advanced

Recurrent kernel machines: computing with infinite echo state networks

Michiel Hermans UGent and Benjamin Schrauwen UGent (2012) NEURAL COMPUTATION. 24(1). p.104-133
abstract
Echo state networks (ESNs) are large, random recurrent neural networks with a single trained linear readout layer. Despite the untrained nature of the recurrent weights, they are capable of performing universal computations on temporal input data, which makes them interesting for both theoretical research and practical applications. The key to their success lies in the fact that the network computes a broad set of nonlinear, spatiotemporal mappings of the input data, on which linear regression or classification can easily be performed. One could consider the reservoir as a spatiotemporal kernel, in which the mapping to a high-dimensional space is computed explicitly. In this letter, we build on this idea and extend the concept of ESNs to infinite-sized recurrent neural networks, which can be considered recursive kernels that subsequently can be used to create recursive support vector machines. We present the theoretical framework, provide several practical examples of recursive kernels, and apply them to typical temporal tasks.
Please use this url to cite or link to this publication:
author
organization
year
type
journalArticle (original)
publication status
published
subject
keyword
MEMORY, TIME-SERIES PREDICTION, SYSTEMS, COMPUTATION, RECOGNITION, NEURAL-NETWORKS
journal title
NEURAL COMPUTATION
Neural Comput.
volume
24
issue
1
pages
104 - 133
Web of Science type
Article
Web of Science id
000297933800005
JCR category
COMPUTER SCIENCE, ARTIFICIAL INTELLIGENCE
JCR impact factor
1.76 (2012)
JCR rank
33/114 (2012)
JCR quartile
2 (2012)
ISSN
0899-7667
DOI
10.1162/NECO_a_00200
language
English
UGent publication?
yes
classification
A1
copyright statement
I have transferred the copyright for this publication to the publisher
id
1979943
handle
http://hdl.handle.net/1854/LU-1979943
date created
2012-01-06 13:08:38
date last changed
2012-01-10 13:34:47
@article{1979943,
  abstract     = {Echo state networks (ESNs) are large, random recurrent neural networks with a single trained linear readout layer. Despite the untrained nature of the recurrent weights, they are capable of performing universal computations on temporal input data, which makes them interesting for both theoretical research and practical applications. The key to their success lies in the fact that the network computes a broad set of nonlinear, spatiotemporal mappings of the input data, on which linear regression or classification can easily be performed. One could consider the reservoir as a spatiotemporal kernel, in which the mapping to a high-dimensional space is computed explicitly. In this letter, we build on this idea and extend the concept of ESNs to infinite-sized recurrent neural networks, which can be considered recursive kernels that subsequently can be used to create recursive support vector machines. We present the theoretical framework, provide several practical examples of recursive kernels, and apply them to typical temporal tasks.},
  author       = {Hermans, Michiel and Schrauwen, Benjamin},
  issn         = {0899-7667},
  journal      = {NEURAL COMPUTATION},
  keyword      = {MEMORY,TIME-SERIES PREDICTION,SYSTEMS,COMPUTATION,RECOGNITION,NEURAL-NETWORKS},
  language     = {eng},
  number       = {1},
  pages        = {104--133},
  title        = {Recurrent kernel machines: computing with infinite echo state networks},
  url          = {http://dx.doi.org/10.1162/NECO\_a\_00200},
  volume       = {24},
  year         = {2012},
}

Chicago
Hermans, Michiel, and Benjamin Schrauwen. 2012. “Recurrent Kernel Machines: Computing with Infinite Echo State Networks.” Neural Computation 24 (1): 104–133.
APA
Hermans, M., & Schrauwen, B. (2012). Recurrent kernel machines: computing with infinite echo state networks. NEURAL COMPUTATION, 24(1), 104–133.
Vancouver
1.
Hermans M, Schrauwen B. Recurrent kernel machines: computing with infinite echo state networks. NEURAL COMPUTATION. 2012;24(1):104–33.
MLA
Hermans, Michiel, and Benjamin Schrauwen. “Recurrent Kernel Machines: Computing with Infinite Echo State Networks.” NEURAL COMPUTATION 24.1 (2012): 104–133. Print.