Academic Bibliography

 Search 200 years of publications by Ghent University researchers.
Advanced search
2 files | 5.64 MB
Add to list
  1. Search results
  2. Fast and reliable uncertainty quantif...

Fast and reliable uncertainty quantification with neural network ensembles for industrial image classification

Arthur Thuy (UGent) and Dries Benoit (UGent)
(2024) ANNALS OF OPERATIONS RESEARCH. 353(2). p.517-543
Author
Organization
Project
Abstract
Image classification with neural networks (NNs) is widely used in industrial processes, situations where the model likely encounters unknown objects during deployment, i.e., out-of-distribution (OOD) data. Worryingly, NNs tend to make confident yet incorrect predictions when confronted with OOD data. To increase the models' reliability, they should quantify the uncertainty in their own predictions, communicating when the output should (not) be trusted. Deep ensembles, composed of multiple independent NNs, have been shown to perform strongly but are computationally expensive. Recent research has proposed more efficient NN ensembles, namely the snapshot, batch, and multi-input multi-output ensemble. This study investigates the predictive and uncertainty performance of efficient NN ensembles in the context of image classification for industrial processes. It is the first to provide a comprehensive comparison and it proposes a novel Diversity Quality metric to quantify the ensembles' performance on the in-distribution and OOD sets in one single metric. The results highlight the batch ensemble as a cost-effective and competitive alternative to the deep ensemble. It matches the deep ensemble in both uncertainty and accuracy while exhibiting considerable savings in training time, test time, and memory storage.
Keywords
Neural network ensembles, Computational efficiency, Uncertainty quantification, Out-of-distribution data, Manufacturing

Downloads

  • Download
    thuy 2024 accepted manuscript.pdf
    • full text (Accepted manuscript)
    • |
    • open access
    • |
    • PDF
    • |
    • 3.26 MB
  • Download
    (...).pdf
    • full text (Published version)
    • |
    • UGent only
    • |
    • PDF
    • |
    • 2.38 MB

Citation

Please use this url to cite or link to this publication:

MLA
Thuy, Arthur, and Dries Benoit. “Fast and Reliable Uncertainty Quantification with Neural Network Ensembles for Industrial Image Classification.” ANNALS OF OPERATIONS RESEARCH, vol. 353, no. 2, 2024, pp. 517–43, doi:10.1007/s10479-024-06440-4.
APA
Thuy, A., & Benoit, D. (2024). Fast and reliable uncertainty quantification with neural network ensembles for industrial image classification. ANNALS OF OPERATIONS RESEARCH, 353(2), 517–543. https://doi.org/10.1007/s10479-024-06440-4
Chicago author-date
Thuy, Arthur, and Dries Benoit. 2024. “Fast and Reliable Uncertainty Quantification with Neural Network Ensembles for Industrial Image Classification.” ANNALS OF OPERATIONS RESEARCH 353 (2): 517–43. https://doi.org/10.1007/s10479-024-06440-4.
Chicago author-date (all authors)
Thuy, Arthur, and Dries Benoit. 2024. “Fast and Reliable Uncertainty Quantification with Neural Network Ensembles for Industrial Image Classification.” ANNALS OF OPERATIONS RESEARCH 353 (2): 517–543. doi:10.1007/s10479-024-06440-4.
Vancouver
1.
Thuy A, Benoit D. Fast and reliable uncertainty quantification with neural network ensembles for industrial image classification. ANNALS OF OPERATIONS RESEARCH. 2024;353(2):517–43.
IEEE
[1]
A. Thuy and D. Benoit, “Fast and reliable uncertainty quantification with neural network ensembles for industrial image classification,” ANNALS OF OPERATIONS RESEARCH, vol. 353, no. 2, pp. 517–543, 2024.
@article{01JHF62WJD2JVEDSCS8DAMMM0S,
  abstract     = {{Image classification with neural networks (NNs) is widely used in industrial processes, situations where the model likely encounters unknown objects during deployment, i.e., out-of-distribution (OOD) data. Worryingly, NNs tend to make confident yet incorrect predictions when confronted with OOD data. To increase the models' reliability, they should quantify the uncertainty in their own predictions, communicating when the output should (not) be trusted. Deep ensembles, composed of multiple independent NNs, have been shown to perform strongly but are computationally expensive. Recent research has proposed more efficient NN ensembles, namely the snapshot, batch, and multi-input multi-output ensemble. This study investigates the predictive and uncertainty performance of efficient NN ensembles in the context of image classification for industrial processes. It is the first to provide a comprehensive comparison and it proposes a novel Diversity Quality metric to quantify the ensembles' performance on the in-distribution and OOD sets in one single metric. The results highlight the batch ensemble as a cost-effective and competitive alternative to the deep ensemble. It matches the deep ensemble in both uncertainty and accuracy while exhibiting considerable savings in training time, test time, and memory storage.}},
  author       = {{Thuy, Arthur and Benoit, Dries}},
  issn         = {{0254-5330}},
  journal      = {{ANNALS OF OPERATIONS RESEARCH}},
  keywords     = {{Neural network ensembles,Computational efficiency,Uncertainty quantification,Out-of-distribution data,Manufacturing}},
  language     = {{eng}},
  number       = {{2}},
  pages        = {{517--543}},
  title        = {{Fast and reliable uncertainty quantification with neural network ensembles for industrial image classification}},
  url          = {{http://doi.org/10.1007/s10479-024-06440-4}},
  volume       = {{353}},
  year         = {{2024}},
}
Altmetric
View in Altmetric
Web of Science
Times cited: