Advanced search
1 file | 626.28 KB

Recurrent kernel machines: computing with infinite echo state networks

Michiel Hermans (UGent) and Benjamin Schrauwen (UGent)
(2012) NEURAL COMPUTATION. 24(1). p.104-133
Author
Organization
Abstract
Echo state networks (ESNs) are large, random recurrent neural networks with a single trained linear readout layer. Despite the untrained nature of the recurrent weights, they are capable of performing universal computations on temporal input data, which makes them interesting for both theoretical research and practical applications. The key to their success lies in the fact that the network computes a broad set of nonlinear, spatiotemporal mappings of the input data, on which linear regression or classification can easily be performed. One could consider the reservoir as a spatiotemporal kernel, in which the mapping to a high-dimensional space is computed explicitly. In this letter, we build on this idea and extend the concept of ESNs to infinite-sized recurrent neural networks, which can be considered recursive kernels that subsequently can be used to create recursive support vector machines. We present the theoretical framework, provide several practical examples of recursive kernels, and apply them to typical temporal tasks.
Keywords
MEMORY, TIME-SERIES PREDICTION, SYSTEMS, COMPUTATION, RECOGNITION, NEURAL-NETWORKS

Downloads

  • Manuscript.pdf
    • full text
    • |
    • open access
    • |
    • PDF
    • |
    • 626.28 KB

Citation

Please use this url to cite or link to this publication:

Chicago
Hermans, Michiel, and Benjamin Schrauwen. 2012. “Recurrent Kernel Machines: Computing with Infinite Echo State Networks.” Neural Computation 24 (1): 104–133.
APA
Hermans, Michiel, & Schrauwen, B. (2012). Recurrent kernel machines: computing with infinite echo state networks. NEURAL COMPUTATION, 24(1), 104–133.
Vancouver
1.
Hermans M, Schrauwen B. Recurrent kernel machines: computing with infinite echo state networks. NEURAL COMPUTATION. 2012;24(1):104–33.
MLA
Hermans, Michiel, and Benjamin Schrauwen. “Recurrent Kernel Machines: Computing with Infinite Echo State Networks.” NEURAL COMPUTATION 24.1 (2012): 104–133. Print.
@article{1979943,
  abstract     = {Echo state networks (ESNs) are large, random recurrent neural networks with a single trained linear readout layer. Despite the untrained nature of the recurrent weights, they are capable of performing universal computations on temporal input data, which makes them interesting for both theoretical research and practical applications. The key to their success lies in the fact that the network computes a broad set of nonlinear, spatiotemporal mappings of the input data, on which linear regression or classification can easily be performed. One could consider the reservoir as a spatiotemporal kernel, in which the mapping to a high-dimensional space is computed explicitly. In this letter, we build on this idea and extend the concept of ESNs to infinite-sized recurrent neural networks, which can be considered recursive kernels that subsequently can be used to create recursive support vector machines. We present the theoretical framework, provide several practical examples of recursive kernels, and apply them to typical temporal tasks.},
  author       = {Hermans, Michiel and Schrauwen, Benjamin},
  issn         = {0899-7667},
  journal      = {NEURAL COMPUTATION},
  keyword      = {MEMORY,TIME-SERIES PREDICTION,SYSTEMS,COMPUTATION,RECOGNITION,NEURAL-NETWORKS},
  language     = {eng},
  number       = {1},
  pages        = {104--133},
  title        = {Recurrent kernel machines: computing with infinite echo state networks},
  url          = {http://dx.doi.org/10.1162/NECO\_a\_00200},
  volume       = {24},
  year         = {2012},
}

Altmetric
View in Altmetric
Web of Science
Times cited: