Advanced search
2 files | 1.56 MB Add to list

A training algorithm for networks of high-variability reservoirs

Matthias Freiberger (UGent) , Peter Bienstman (UGent) and Joni Dambre (UGent)
Author
Organization
Abstract
Physical reservoir computing approaches have gained increased attention in recent years due to their potential for low-energy high-performance computing. Despite recent successes, there are bounds to what one can achieve simply by making physical reservoirs larger. Therefore, we argue that a switch from single-reservoir computing to multi-reservoir and even deep physical reservoir computing is desirable. Given that error backpropagation cannot be used directly to train a large class of multi-reservoir systems, we propose an alternative framework that combines the power of backpropagation with the speed and simplicity of classic training algorithms. In this work we report our findings on a conducted experiment to evaluate the general feasibility of our approach. We train a network of 3 Echo State Networks to perform the well-known NARMA-10 task, where we use intermediate targets derived through backpropagation. Our results indicate that our proposed method is well-suited to train multi-reservoir systems in an efficient way.
Keywords
deep reservoir computing, target signal derivation, multi-reservoir architectures, training algorithms, recurrent neural networks, NEURAL-NETWORKS, SYSTEMS

Downloads

  • s41598-020-71549-y 1 .pdf
    • full text (Published version)
    • |
    • open access
    • |
    • PDF
    • |
    • 1.28 MB
  • 41598 2020 71549 MOESM1 ESM.pdf
    • supplementary material
    • |
    • open access
    • |
    • PDF
    • |
    • 279.45 KB

Citation

Please use this url to cite or link to this publication:

MLA
Freiberger, Matthias, et al. “A Training Algorithm for Networks of High-Variability Reservoirs.” SCIENTIFIC REPORTS, vol. 10, no. 1, 2020, doi:10.1038/s41598-020-71549-y.
APA
Freiberger, M., Bienstman, P., & Dambre, J. (2020). A training algorithm for networks of high-variability reservoirs. SCIENTIFIC REPORTS, 10(1). https://doi.org/10.1038/s41598-020-71549-y
Chicago author-date
Freiberger, Matthias, Peter Bienstman, and Joni Dambre. 2020. “A Training Algorithm for Networks of High-Variability Reservoirs.” SCIENTIFIC REPORTS 10 (1). https://doi.org/10.1038/s41598-020-71549-y.
Chicago author-date (all authors)
Freiberger, Matthias, Peter Bienstman, and Joni Dambre. 2020. “A Training Algorithm for Networks of High-Variability Reservoirs.” SCIENTIFIC REPORTS 10 (1). doi:10.1038/s41598-020-71549-y.
Vancouver
1.
Freiberger M, Bienstman P, Dambre J. A training algorithm for networks of high-variability reservoirs. SCIENTIFIC REPORTS. 2020;10(1).
IEEE
[1]
M. Freiberger, P. Bienstman, and J. Dambre, “A training algorithm for networks of high-variability reservoirs,” SCIENTIFIC REPORTS, vol. 10, no. 1, 2020.
@article{8692664,
  abstract     = {{Physical reservoir computing approaches have gained increased attention in recent years due to their potential for low-energy high-performance computing. Despite recent successes, there are bounds to what one can achieve simply by making physical reservoirs larger. Therefore, we argue that a switch from single-reservoir computing to multi-reservoir and even deep physical reservoir computing is desirable. Given that error backpropagation cannot be used directly to train a large class of multi-reservoir systems, we propose an alternative framework that combines the power of backpropagation with the speed and simplicity of classic training algorithms. In this work we report our findings on a conducted experiment to evaluate the general feasibility of our approach. We train a network of 3 Echo State Networks to perform the well-known NARMA-10 task, where we use intermediate targets derived through backpropagation. Our results indicate that our proposed method is well-suited to train multi-reservoir systems in an efficient way.}},
  articleno    = {{14451}},
  author       = {{Freiberger, Matthias and Bienstman, Peter and Dambre, Joni}},
  issn         = {{2045-2322}},
  journal      = {{SCIENTIFIC REPORTS}},
  keywords     = {{deep reservoir computing,target signal derivation,multi-reservoir architectures,training algorithms,recurrent neural networks,NEURAL-NETWORKS,SYSTEMS}},
  language     = {{eng}},
  number       = {{1}},
  pages        = {{11}},
  title        = {{A training algorithm for networks of high-variability reservoirs}},
  url          = {{http://doi.org/10.1038/s41598-020-71549-y}},
  volume       = {{10}},
  year         = {{2020}},
}

Altmetric
View in Altmetric
Web of Science
Times cited: