Advanced search
2 files | 4.03 MB Add to list

Valid prediction intervals for regression problems

Nicolas Dewolf (UGent) , Bernard De Baets (UGent) and Willem Waegeman (UGent)
(2023) ARTIFICIAL INTELLIGENCE REVIEW. 56(1). p.577-613
Author
Organization
Project
Abstract
Over the last few decades, various methods have been proposed for estimating prediction intervals in regression settings, including Bayesian methods, ensemble methods, direct interval estimation methods and conformal prediction methods. An important issue is the validity and calibration of these methods: the generated prediction intervals should have a predefined coverage level, without being overly conservative. So far, no study has analysed this issue whilst simultaneously considering these four classes of methods. In this independent comparative study, we review the above four classes of methods from a conceptual and experimental point of view in the i.i.d. setting. Results on benchmark data sets from various domains highlight large fluctuations in performance from one data set to another. These observations can be attributed to the violation of certain assumptions that are inherent to some classes of methods. We illustrate how conformal prediction can be used as a general calibration procedure for methods that deliver poor results without a calibration step.
Keywords
Conformal prediction, Ensemble theory, Bayesian network, Regression, Calibration, Prediction interval

Downloads

  • (...).pdf
    • full text (Published version)
    • |
    • UGent only
    • |
    • PDF
    • |
    • 2.76 MB
  • Kermit A1 702 accepted.pdf
    • full text (Accepted manuscript)
    • |
    • open access
    • |
    • PDF
    • |
    • 1.27 MB

Citation

Please use this url to cite or link to this publication:

MLA
Dewolf, Nicolas, et al. “Valid Prediction Intervals for Regression Problems.” ARTIFICIAL INTELLIGENCE REVIEW, vol. 56, no. 1, 2023, pp. 577–613, doi:10.1007/s10462-022-10178-5.
APA
Dewolf, N., De Baets, B., & Waegeman, W. (2023). Valid prediction intervals for regression problems. ARTIFICIAL INTELLIGENCE REVIEW, 56(1), 577–613. https://doi.org/10.1007/s10462-022-10178-5
Chicago author-date
Dewolf, Nicolas, Bernard De Baets, and Willem Waegeman. 2023. “Valid Prediction Intervals for Regression Problems.” ARTIFICIAL INTELLIGENCE REVIEW 56 (1): 577–613. https://doi.org/10.1007/s10462-022-10178-5.
Chicago author-date (all authors)
Dewolf, Nicolas, Bernard De Baets, and Willem Waegeman. 2023. “Valid Prediction Intervals for Regression Problems.” ARTIFICIAL INTELLIGENCE REVIEW 56 (1): 577–613. doi:10.1007/s10462-022-10178-5.
Vancouver
1.
Dewolf N, De Baets B, Waegeman W. Valid prediction intervals for regression problems. ARTIFICIAL INTELLIGENCE REVIEW. 2023;56(1):577–613.
IEEE
[1]
N. Dewolf, B. De Baets, and W. Waegeman, “Valid prediction intervals for regression problems,” ARTIFICIAL INTELLIGENCE REVIEW, vol. 56, no. 1, pp. 577–613, 2023.
@article{01GRKBKYA09A1N2AS3Y0K7015T,
  abstract     = {{Over the last few decades, various methods have been proposed for estimating prediction intervals in regression settings, including Bayesian methods, ensemble methods, direct interval estimation methods and conformal prediction methods. An important issue is the validity and calibration of these methods: the generated prediction intervals should have a predefined coverage level, without being overly conservative. So far, no study has analysed this issue whilst simultaneously considering these four classes of methods. In this independent comparative study, we review the above four classes of methods from a conceptual and experimental point of view in the i.i.d. setting. Results on benchmark data sets from various domains highlight large fluctuations in performance from one data set to another. These observations can be attributed to the violation of certain assumptions that are inherent to some classes of methods. We illustrate how conformal prediction can be used as a general calibration procedure for methods that deliver poor results without a calibration step.}},
  author       = {{Dewolf, Nicolas and De Baets, Bernard and Waegeman, Willem}},
  issn         = {{0269-2821}},
  journal      = {{ARTIFICIAL INTELLIGENCE REVIEW}},
  keywords     = {{Conformal prediction,Ensemble theory,Bayesian network,Regression,Calibration,Prediction interval}},
  language     = {{eng}},
  number       = {{1}},
  pages        = {{577--613}},
  title        = {{Valid prediction intervals for regression problems}},
  url          = {{http://doi.org/10.1007/s10462-022-10178-5}},
  volume       = {{56}},
  year         = {{2023}},
}

Altmetric
View in Altmetric
Web of Science
Times cited: