Advanced search
2 files | 6.63 MB Add to list

Asynchronous spiking neurons, the natural key to exploit temporal sparsity

Author
Organization
Abstract
Inference of Deep Neural Networks for stream signal (Video/Audio) processing in edge devices is still challenging. Unlike the most state of the art inference engines which are efficient for static signals, our brain is optimized for real-time dynamic signal processing. We believe one important feature of the brain (asynchronous state-full processing) is the key to its excellence in this domain. In this work, we show how asynchronous processing with state-full neurons allows exploitation of the existing sparsity in natural signals. This paper explains three different types of sparsity and proposes an inference algorithm which exploits all types of sparsities in the execution of already trained networks. Our experiments in three different applications (Handwritten digit recognition, Autonomous Steering and Hand-Gesture recognition) show that this model of inference reduces the number of required operations for sparse input data by a factor of one to two orders of magnitudes. Additionally, due to fully asynchronous processing this type of inference can be run on fully distributed and scalable neuromorphic hardware platforms.
Keywords
NETWORKS, BRAIN, Neural networks, Artificial neural networks, Inference algorithms, Algorithm design and analysis, Asynchronous communication, Spiking, Neural Network (SNN), Asynchronous Inference, Temporal sparsity, Deep, Neural Network, Convolutional Neural Network (CNN), Bio-inspired processing, Neuromorphic Hardware

Downloads

  • (...).pdf
    • full text (Published version)
    • |
    • UGent only
    • |
    • PDF
    • |
    • 4.27 MB
  • Yousefzadeh A accepted version.pdf
    • full text (Accepted manuscript)
    • |
    • open access
    • |
    • PDF
    • |
    • 2.36 MB

Citation

Please use this url to cite or link to this publication:

MLA
Yousefzadeh, Amirreza, et al. “Asynchronous Spiking Neurons, the Natural Key to Exploit Temporal Sparsity.” IEEE JOURNAL ON EMERGING AND SELECTED TOPICS IN CIRCUITS AND SYSTEMS, vol. 9, no. 4, 2019, pp. 668–78.
APA
Yousefzadeh, A., Khoei, M. A., Hoseini, S., Cavalcante Holanda, P., Leroux, S., Moreira, O., … Linares-Barranco, B. (2019). Asynchronous spiking neurons, the natural key to exploit temporal sparsity. IEEE JOURNAL ON EMERGING AND SELECTED TOPICS IN CIRCUITS AND SYSTEMS, 9(4), 668–678.
Chicago author-date
Yousefzadeh, Amirreza, Mina A. Khoei, Sahar Hoseini, Priscila Cavalcante Holanda, Sam Leroux, Orlando Moreira, Jonathan Tapson, et al. 2019. “Asynchronous Spiking Neurons, the Natural Key to Exploit Temporal Sparsity.” IEEE JOURNAL ON EMERGING AND SELECTED TOPICS IN CIRCUITS AND SYSTEMS 9 (4): 668–78.
Chicago author-date (all authors)
Yousefzadeh, Amirreza, Mina A. Khoei, Sahar Hoseini, Priscila Cavalcante Holanda, Sam Leroux, Orlando Moreira, Jonathan Tapson, Bart Dhoedt, Pieter Simoens, Teresa Serrano-Gotarredona, and Bernabe Linares-Barranco. 2019. “Asynchronous Spiking Neurons, the Natural Key to Exploit Temporal Sparsity.” IEEE JOURNAL ON EMERGING AND SELECTED TOPICS IN CIRCUITS AND SYSTEMS 9 (4): 668–678.
Vancouver
1.
Yousefzadeh A, Khoei MA, Hoseini S, Cavalcante Holanda P, Leroux S, Moreira O, et al. Asynchronous spiking neurons, the natural key to exploit temporal sparsity. IEEE JOURNAL ON EMERGING AND SELECTED TOPICS IN CIRCUITS AND SYSTEMS. 2019;9(4):668–78.
IEEE
[1]
A. Yousefzadeh et al., “Asynchronous spiking neurons, the natural key to exploit temporal sparsity,” IEEE JOURNAL ON EMERGING AND SELECTED TOPICS IN CIRCUITS AND SYSTEMS, vol. 9, no. 4, pp. 668–678, 2019.
@article{8640460,
  abstract     = {Inference of Deep Neural Networks for stream signal (Video/Audio) processing in edge devices is still challenging. Unlike the most state of the art inference engines which are efficient for static signals, our brain is optimized for real-time dynamic signal processing. We believe one important feature of the brain (asynchronous state-full processing) is the key to its excellence in this domain. In this work, we show how asynchronous processing with state-full neurons allows exploitation of the existing sparsity in natural signals. This paper explains three different types of sparsity and proposes an inference algorithm which exploits all types of sparsities in the execution of already trained networks. Our experiments in three different applications (Handwritten digit recognition, Autonomous Steering and Hand-Gesture recognition) show that this model of inference reduces the number of required operations for sparse input data by a factor of one to two orders of magnitudes. Additionally, due to fully asynchronous processing this type of inference can be run on fully distributed and scalable neuromorphic hardware platforms.},
  author       = {Yousefzadeh, Amirreza and Khoei, Mina A. and Hoseini, Sahar and Cavalcante Holanda, Priscila and Leroux, Sam and Moreira, Orlando and Tapson, Jonathan and Dhoedt, Bart and Simoens, Pieter and Serrano-Gotarredona, Teresa and Linares-Barranco, Bernabe},
  issn         = {2156-3357},
  journal      = {IEEE JOURNAL ON EMERGING AND SELECTED TOPICS IN CIRCUITS AND SYSTEMS},
  keywords     = {NETWORKS,BRAIN,Neural networks,Artificial neural networks,Inference algorithms,Algorithm design and analysis,Asynchronous communication,Spiking,Neural Network (SNN),Asynchronous Inference,Temporal sparsity,Deep,Neural Network,Convolutional Neural Network (CNN),Bio-inspired processing,Neuromorphic Hardware},
  language     = {eng},
  location     = {Hsinchu, Taiwan},
  number       = {4},
  pages        = {668--678},
  title        = {Asynchronous spiking neurons, the natural key to exploit temporal sparsity},
  url          = {http://dx.doi.org/10.1109/JETCAS.2019.2951121},
  volume       = {9},
  year         = {2019},
}

Altmetric
View in Altmetric
Web of Science
Times cited: