Learnable axonal delay in spiking neural networks improves spoken word recognition
- Author
- Pengfei Sun (UGent) , Yansong Chua, Paul Devos (UGent) and Dick Botteldooren (UGent)
- Organization
- Project
- Abstract
- Spiking neural networks (SNNs), which are composed of biologically plausible spiking neurons, and combined with bio-physically realistic auditory periphery models, offer a means to explore and understand human auditory processing-especially in tasks where precise timing is essential. However, because of the inherent temporal complexity in spike sequences, the performance of SNNs has remained less competitive compared to artificial neural networks (ANNs). To tackle this challenge, a fundamental research topic is the configuration of spike-timing and the exploration of more intricate architectures. In this work, we demonstrate a learnable axonal delay combined with local skip-connections yields state-of-the-art performance on challenging benchmarks for spoken word recognition. Additionally, we introduce an auxiliary loss term to further enhance accuracy and stability. Experiments on the neuromorphic speech benchmark datasets, NTIDIDIGITS and SHD, show improvements in performance when incorporating our delay module in comparison to vanilla feedforward SNNs. Specifically, with the integration of our delay module, the performance on NTIDIDIGITS and SHD improves by 14% and 18%, respectively. When paired with local skip-connections and the auxiliary loss, our approach surpasses both recurrent and convolutional neural networks, yet uses 10 x fewer parameters for NTIDIDIGITS and 7 x fewer for SHD.
- Keywords
- BACKPROPAGATION, axonal delay, spiking neural network, speech processing, supervised learning, auditory modeling, neuromorphic computing
Downloads
-
ACUS 664.pdf
- full text (Published version)
- |
- open access
- |
- |
- 2.06 MB
Citation
Please use this url to cite or link to this publication: http://hdl.handle.net/1854/LU-01HJ19WCENS3R3HF931MMTBR3K
- MLA
- Sun, Pengfei, et al. “Learnable Axonal Delay in Spiking Neural Networks Improves Spoken Word Recognition.” FRONTIERS IN NEUROSCIENCE, vol. 17, 2023, doi:10.3389/fnins.2023.1275944.
- APA
- Sun, P., Chua, Y., Devos, P., & Botteldooren, D. (2023). Learnable axonal delay in spiking neural networks improves spoken word recognition. FRONTIERS IN NEUROSCIENCE, 17. https://doi.org/10.3389/fnins.2023.1275944
- Chicago author-date
- Sun, Pengfei, Yansong Chua, Paul Devos, and Dick Botteldooren. 2023. “Learnable Axonal Delay in Spiking Neural Networks Improves Spoken Word Recognition.” FRONTIERS IN NEUROSCIENCE 17. https://doi.org/10.3389/fnins.2023.1275944.
- Chicago author-date (all authors)
- Sun, Pengfei, Yansong Chua, Paul Devos, and Dick Botteldooren. 2023. “Learnable Axonal Delay in Spiking Neural Networks Improves Spoken Word Recognition.” FRONTIERS IN NEUROSCIENCE 17. doi:10.3389/fnins.2023.1275944.
- Vancouver
- 1.Sun P, Chua Y, Devos P, Botteldooren D. Learnable axonal delay in spiking neural networks improves spoken word recognition. FRONTIERS IN NEUROSCIENCE. 2023;17.
- IEEE
- [1]P. Sun, Y. Chua, P. Devos, and D. Botteldooren, “Learnable axonal delay in spiking neural networks improves spoken word recognition,” FRONTIERS IN NEUROSCIENCE, vol. 17, 2023.
@article{01HJ19WCENS3R3HF931MMTBR3K,
abstract = {{Spiking neural networks (SNNs), which are composed of biologically plausible spiking neurons, and combined with bio-physically realistic auditory periphery models, offer a means to explore and understand human auditory processing-especially in tasks where precise timing is essential. However, because of the inherent temporal complexity in spike sequences, the performance of SNNs has remained less competitive compared to artificial neural networks (ANNs). To tackle this challenge, a fundamental research topic is the configuration of spike-timing and the exploration of more intricate architectures. In this work, we demonstrate a learnable axonal delay combined with local skip-connections yields state-of-the-art performance on challenging benchmarks for spoken word recognition. Additionally, we introduce an auxiliary loss term to further enhance accuracy and stability. Experiments on the neuromorphic speech benchmark datasets, NTIDIDIGITS and SHD, show improvements in performance when incorporating our delay module in comparison to vanilla feedforward SNNs. Specifically, with the integration of our delay module, the performance on NTIDIDIGITS and SHD improves by 14% and 18%, respectively. When paired with local skip-connections and the auxiliary loss, our approach surpasses both recurrent and convolutional neural networks, yet uses 10 x fewer parameters for NTIDIDIGITS and 7 x fewer for SHD.}},
articleno = {{1275944}},
author = {{Sun, Pengfei and Chua, Yansong and Devos, Paul and Botteldooren, Dick}},
issn = {{1662-453X}},
journal = {{FRONTIERS IN NEUROSCIENCE}},
keywords = {{BACKPROPAGATION,axonal delay,spiking neural network,speech processing,supervised learning,auditory modeling,neuromorphic computing}},
language = {{eng}},
pages = {{12}},
title = {{Learnable axonal delay in spiking neural networks improves spoken word recognition}},
url = {{http://doi.org/10.3389/fnins.2023.1275944}},
volume = {{17}},
year = {{2023}},
}
- Altmetric
- View in Altmetric
- Web of Science
- Times cited: