- Author
- Arne Gevaert (UGent) , Axel-Jan Rousseau, Thijs Becker, Dirk Valkenborg, Tijl De Bie (UGent) and Yvan Saeys (UGent)
- Organization
- Abstract
- Feature attribution maps are a popular approach to highlight the most important pixels in an image for a given prediction of a model. Despite a recent growth in popularity and available methods, little attention is given to the objective evaluation of such attribution maps. Building on previous work in this domain, we investigate existing metrics and propose new variants of metrics for the evaluation of attribution maps. We confirm a recent finding that different attribution metrics seem to measure different underlying concepts of attribution maps, and extend this finding to a larger selection of attribution metrics. We also find that metric results on one dataset do not necessarily generalize to other datasets, and methods with desirable theoretical properties such as DeepSHAP do not necessarily outperform computationally cheaper alternatives. Based on these findings, we propose a general benchmarking approach to identify the ideal feature attribution method for a given use case. Implementations of attribution metrics and our experiments are available online.
Downloads
-
2202.12270.pdf
- full text (Author's original)
- |
- open access
- |
- |
- 7.43 MB
Citation
Please use this url to cite or link to this publication: http://hdl.handle.net/1854/LU-01GT9PD0DTE866DMPR8GJZWG0Z
- MLA
- Gevaert, Arne, et al. “Evaluating Feature Attribution Methods in the Image Domain.” ArXiv, 2022, doi:10.48550/arXiv.2202.12270.
- APA
- Gevaert, A., Rousseau, A.-J., Becker, T., Valkenborg, D., De Bie, T., & Saeys, Y. (2022). Evaluating feature attribution methods in the image domain. https://doi.org/10.48550/arXiv.2202.12270
- Chicago author-date
- Gevaert, Arne, Axel-Jan Rousseau, Thijs Becker, Dirk Valkenborg, Tijl De Bie, and Yvan Saeys. 2022. “Evaluating Feature Attribution Methods in the Image Domain.” ArXiv. https://doi.org/10.48550/arXiv.2202.12270.
- Chicago author-date (all authors)
- Gevaert, Arne, Axel-Jan Rousseau, Thijs Becker, Dirk Valkenborg, Tijl De Bie, and Yvan Saeys. 2022. “Evaluating Feature Attribution Methods in the Image Domain.” ArXiv. doi:10.48550/arXiv.2202.12270.
- Vancouver
- 1.Gevaert A, Rousseau A-J, Becker T, Valkenborg D, De Bie T, Saeys Y. Evaluating feature attribution methods in the image domain. ArXiv. 2022.
- IEEE
- [1]A. Gevaert, A.-J. Rousseau, T. Becker, D. Valkenborg, T. De Bie, and Y. Saeys, “Evaluating feature attribution methods in the image domain,” ArXiv. 2022.
@misc{01GT9PD0DTE866DMPR8GJZWG0Z,
abstract = {{ Feature attribution maps are a popular approach to highlight the most
important pixels in an image for a given prediction of a model. Despite a
recent growth in popularity and available methods, little attention is given to
the objective evaluation of such attribution maps. Building on previous work in
this domain, we investigate existing metrics and propose new variants of
metrics for the evaluation of attribution maps. We confirm a recent finding
that different attribution metrics seem to measure different underlying
concepts of attribution maps, and extend this finding to a larger selection of
attribution metrics. We also find that metric results on one dataset do not
necessarily generalize to other datasets, and methods with desirable
theoretical properties such as DeepSHAP do not necessarily outperform
computationally cheaper alternatives. Based on these findings, we propose a
general benchmarking approach to identify the ideal feature attribution method
for a given use case. Implementations of attribution metrics and our
experiments are available online.
}},
author = {{Gevaert, Arne and Rousseau, Axel-Jan and Becker, Thijs and Valkenborg, Dirk and De Bie, Tijl and Saeys, Yvan}},
language = {{eng}},
pages = {{38}},
series = {{ArXiv}},
title = {{Evaluating feature attribution methods in the image domain}},
url = {{http://doi.org/10.48550/arXiv.2202.12270}},
year = {{2022}},
}
- Altmetric
- View in Altmetric