
Spectral feature fusion networks with dual attention for hyperspectral image classification
- Author
- Xian Li, Mingli Ding and Aleksandra Pizurica (UGent)
- Organization
- Abstract
- Recent progress in spectral classification is largely attributed to the use of convolutional neural networks (CNN). While a variety of successful architectures have been proposed, they all extract spectral features from various portions of adjacent spectral bands. In this paper, we take a different approach and develop a deep spectral feature fusion method, which extracts both local and interlocal spectral features, capturing thus also the correlations among non-adjacent bands. To our knowledge, this is the first reported deep spectral feature fusion method. Our model is a two-stream architecture, where an intergroup and a groupwise spectral classifiers operate in parallel. The interlocal spectral correlation feature extraction is achieved elegantly, by reshaping the input spectral vectors to form the socalled non-adjacent spectral matrices. We introduce the concept of groupwise band convolution to enable efficient extraction of discriminative local features with multiple kernels adopting to the local spectral content. Another important contribution of this work is a novel dual-channel attention mechanism to identify the most informative spectral features. The model is trained in an end-to-end fashion with a joint loss. Experimental results on real data sets demonstrate excellent performance compared to the current state-of-the-art.
- Keywords
- Electrical and Electronic Engineering, General Earth and Planetary Sciences, Feature extraction, Correlation, Hyperspectral imaging, Training, Testing, Convolution, Deep learning, Attention mechanism, deep learning, hyperspectral image classification, spectral feature fusion, MARKOV-RANDOM-FIELDS, CNN
Downloads
-
TGRS-2021-00131.R2.pdf
- full text (Accepted manuscript)
- |
- open access
- |
- |
- 1.84 MB
-
(...).pdf
- full text (Published version)
- |
- UGent only
- |
- |
- 11.09 MB
Citation
Please use this url to cite or link to this publication: http://hdl.handle.net/1854/LU-8711434
- MLA
- Li, Xian, et al. “Spectral Feature Fusion Networks with Dual Attention for Hyperspectral Image Classification.” IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING, vol. 60, 2022, doi:10.1109/tgrs.2021.3084922.
- APA
- Li, X., Ding, M., & Pizurica, A. (2022). Spectral feature fusion networks with dual attention for hyperspectral image classification. IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING, 60. https://doi.org/10.1109/tgrs.2021.3084922
- Chicago author-date
- Li, Xian, Mingli Ding, and Aleksandra Pizurica. 2022. “Spectral Feature Fusion Networks with Dual Attention for Hyperspectral Image Classification.” IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING 60. https://doi.org/10.1109/tgrs.2021.3084922.
- Chicago author-date (all authors)
- Li, Xian, Mingli Ding, and Aleksandra Pizurica. 2022. “Spectral Feature Fusion Networks with Dual Attention for Hyperspectral Image Classification.” IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING 60. doi:10.1109/tgrs.2021.3084922.
- Vancouver
- 1.Li X, Ding M, Pizurica A. Spectral feature fusion networks with dual attention for hyperspectral image classification. IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING. 2022;60.
- IEEE
- [1]X. Li, M. Ding, and A. Pizurica, “Spectral feature fusion networks with dual attention for hyperspectral image classification,” IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING, vol. 60, 2022.
@article{8711434, abstract = {{Recent progress in spectral classification is largely attributed to the use of convolutional neural networks (CNN). While a variety of successful architectures have been proposed, they all extract spectral features from various portions of adjacent spectral bands. In this paper, we take a different approach and develop a deep spectral feature fusion method, which extracts both local and interlocal spectral features, capturing thus also the correlations among non-adjacent bands. To our knowledge, this is the first reported deep spectral feature fusion method. Our model is a two-stream architecture, where an intergroup and a groupwise spectral classifiers operate in parallel. The interlocal spectral correlation feature extraction is achieved elegantly, by reshaping the input spectral vectors to form the socalled non-adjacent spectral matrices. We introduce the concept of groupwise band convolution to enable efficient extraction of discriminative local features with multiple kernels adopting to the local spectral content. Another important contribution of this work is a novel dual-channel attention mechanism to identify the most informative spectral features. The model is trained in an end-to-end fashion with a joint loss. Experimental results on real data sets demonstrate excellent performance compared to the current state-of-the-art.}}, articleno = {{5508614}}, author = {{Li, Xian and Ding, Mingli and Pizurica, Aleksandra}}, issn = {{0196-2892}}, journal = {{IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING}}, keywords = {{Electrical and Electronic Engineering,General Earth and Planetary Sciences,Feature extraction,Correlation,Hyperspectral imaging,Training,Testing,Convolution,Deep learning,Attention mechanism,deep learning,hyperspectral image classification,spectral feature fusion,MARKOV-RANDOM-FIELDS,CNN}}, language = {{eng}}, pages = {{14}}, title = {{Spectral feature fusion networks with dual attention for hyperspectral image classification}}, url = {{http://doi.org/10.1109/tgrs.2021.3084922}}, volume = {{60}}, year = {{2022}}, }
- Altmetric
- View in Altmetric
- Web of Science
- Times cited: