Advanced search
1 file | 74.19 KB Add to list

One step backpropagation through time for learning input mapping in reservoir computing applied to speech recognition

Michiel Hermans (UGent) and Benjamin Schrauwen (UGent)
Author
Organization
Abstract
Recurrent neural networks are very powerful engines for processing information that is coded in time, however, many problems with common training algorithms, such as Backpropagation Through Time, remain. Because of this, another important learning setup known as Reservoir Computing has appeared in recent years, where one uses an essentially untrained network to perform computations. Though very successful in many applications, using a random network can be quite inefficient when considering the required number of neurons and the associated computational costs. In this paper we introduce a highly simplified version of Backpropagation Through Time by basically truncating the error backpropagation to one step back in time, and we combine this with the classic Reservoir Computing setup using an instantaneous linear readout. We apply this setup to a spoken digit recognition task and show it to give very good results for small networks.
Keywords
speech recognition, backpropagation, learning (artificial intelligence), recurrent neural nets, NEURAL-NETWORKS, SYSTEMS

Downloads

  • ISCAS2010.pdf
    • full text
    • |
    • open access
    • |
    • PDF
    • |
    • 74.19 KB

Citation

Please use this url to cite or link to this publication:

MLA
Hermans, Michiel, and Benjamin Schrauwen. “One Step Backpropagation through Time for Learning Input Mapping in Reservoir Computing Applied to Speech Recognition.” IEEE International Symposium on Circuits and Systems, IEEE, 2010, pp. 521–24, doi:10.1109/ISCAS.2010.5537568.
APA
Hermans, M., & Schrauwen, B. (2010). One step backpropagation through time for learning input mapping in reservoir computing applied to speech recognition. IEEE International Symposium on Circuits and Systems, 521–524. https://doi.org/10.1109/ISCAS.2010.5537568
Chicago author-date
Hermans, Michiel, and Benjamin Schrauwen. 2010. “One Step Backpropagation through Time for Learning Input Mapping in Reservoir Computing Applied to Speech Recognition.” In IEEE International Symposium on Circuits and Systems, 521–24. New York, NY, USA: IEEE. https://doi.org/10.1109/ISCAS.2010.5537568.
Chicago author-date (all authors)
Hermans, Michiel, and Benjamin Schrauwen. 2010. “One Step Backpropagation through Time for Learning Input Mapping in Reservoir Computing Applied to Speech Recognition.” In IEEE International Symposium on Circuits and Systems, 521–524. New York, NY, USA: IEEE. doi:10.1109/ISCAS.2010.5537568.
Vancouver
1.
Hermans M, Schrauwen B. One step backpropagation through time for learning input mapping in reservoir computing applied to speech recognition. In: IEEE International Symposium on Circuits and Systems. New York, NY, USA: IEEE; 2010. p. 521–4.
IEEE
[1]
M. Hermans and B. Schrauwen, “One step backpropagation through time for learning input mapping in reservoir computing applied to speech recognition,” in IEEE International Symposium on Circuits and Systems, Paris, France, 2010, pp. 521–524.
@inproceedings{1065982,
  abstract     = {{Recurrent neural networks are very powerful engines for processing information that is coded in time, however, many problems with common training algorithms, such as Backpropagation Through Time, remain. Because of this, another important learning setup known as Reservoir Computing has appeared in recent years, where one uses an essentially untrained network to perform computations. Though very successful in many applications, using a random network can be quite inefficient when considering the required number of neurons and the associated computational costs. In this paper we introduce a highly simplified version of Backpropagation Through Time by basically truncating the error backpropagation to one step back in time, and we combine this with the classic Reservoir Computing setup using an instantaneous linear readout. We apply this setup to a spoken digit recognition task and show it to give very good results for small networks.}},
  author       = {{Hermans, Michiel and Schrauwen, Benjamin}},
  booktitle    = {{IEEE International Symposium on Circuits and Systems}},
  isbn         = {{9781424453092}},
  issn         = {{0271-4302}},
  keywords     = {{speech recognition,backpropagation,learning (artificial intelligence),recurrent neural nets,NEURAL-NETWORKS,SYSTEMS}},
  language     = {{eng}},
  location     = {{Paris, France}},
  pages        = {{521--524}},
  publisher    = {{IEEE}},
  title        = {{One step backpropagation through time for learning input mapping in reservoir computing applied to speech recognition}},
  url          = {{http://doi.org/10.1109/ISCAS.2010.5537568}},
  year         = {{2010}},
}

Altmetric
View in Altmetric
Web of Science
Times cited: